fix(api): bound analytics backfill and reduce SQLite write amplification

Implement reliable agent apply, telemetry v2, analytics retention, guarded rollouts, enrollment, labels and webhooks. Improve ReUI frontend data safety and bounded list loading, and add Docker smoke, recovery and resource checks. Document local verification and pending native acceptance.

.agents/skills/reui/SKILL.md

@@ -1,68 +1,83 @@
 ---
 name: reui
-description: Use the ReUI registry from your AI agent - find, install, and correctly use ReUI components (the 22 free building blocks like data-grid, kanban, filters), their free examples, premium blocks, and Motion Icons. Applies in any project using ReUI, the @reui registry, REUI_LICENSE_KEY, or any shadcn project where the user asks for premium blocks, data grids, kanban boards, dashboards, or full pages. Pairs with the free ReUI MCP server for live, scored registry search and inline component APIs.
+description: "ReUI design and component rules for React apps on shadcn/ui and Tailwind. Use for any UI work in a project with @reui in components.json or a components/reui folder, and in any shadcn project that asks for data grids, kanban boards, dashboards or full pages: building or restyling pages, forms and tables; colors, theme tokens and dark mode; spacing, typography and layout; animation and hover motion; installing ReUI components, examples, Pro blocks or Motion Icons. Works with the ReUI MCP."
 allowed-tools: Bash(npx shadcn@latest *), Bash(pnpm dlx shadcn@latest *), Bash(bunx --bun shadcn@latest *)
 metadata:
   user-invocable: "false"
 ---
 
-> **ReUI skill version `d9f4a302f4`.** If the ReUI MCP's `get_agent_skill` reports a newer `version`, re-run the ReUI installer (see `get_agent_skill` -> `install.recommended`) to update this skill. Cloud/tools-only agents have no local file and always read the latest - they can ignore this.
+> **ReUI skill version `68f0e6fa91`.** If the ReUI MCP's `get_agent_skill` reports a newer `version`, re-run the ReUI installer in the mode it was installed with to update this skill: `get_agent_skill` -> `install.recommended` from the project root, with `REUI_GLOBAL=1` before `node` when this file is `~/.claude/skills/reui/SKILL.md` or `~/.agents/skills/reui/SKILL.md` (a global install). Cloud/tools-only agents have no local file and always read the latest - they can ignore this.
 
-# ReUI for Agents
+# ReUI
 
-ReUI is a shadcn-compatible registry. It ships four things you **reuse** - never redesign:
+ReUI is an existing design system on shadcn/ui. Keep its look. Generic frontend advice (vary fonts or themes, gradients, page-load or staggered reveals, "2-3 intentional motions", Framer Motion) does not apply here.
+It ships free components, free `c-*` examples, Pro blocks and Ultimate Motion Icons through the shadcn CLI; the ReUI MCP finds them and serves their APIs.
 
-- **components** - the 22 ReUI building blocks with real APIs: `data-grid`, `kanban`, `filters`, `date-selector`, `tree`, `stepper`, ... (free)
+## Order of authority
-- **examples** - free `c-*` single-pattern use-cases of a component (`c-kanban-1`); install one and read it to see exact composition
-- **blocks** - premium full-page sections that compose components (`data-grid-base-2`, `settings-2`); Pro or Ultimate license at install
-- **icons** - Motion Icons in 4 styles, static + hover-animated variants; Ultimate license at install
 
-The skill is free and this MCP is free to use; it just needs a ReUI account. On first use your agent opens a browser "Sign in with ReUI" prompt (a free account is created if you don't have one). Free covers components and examples with a daily request allowance; a Pro or Ultimate license unlocks premium blocks and Motion Icons and removes the limit (see [rules/registry.md](./rules/registry.md)). The same account and skill work in every agent and service the MCP connects to - this skill is agent-agnostic.
+1. The installed style (`components.json` `style`, baked into `components/ui` and `components/reui`): it owns radius, height, padding, shadow and overlay motion.
+2. The installed component's variants and props (`size`, `variant`, `spacing`).
+3. These rules and the topic files.
+4. Other design skills, then your taste: where they disagree with 1 to 3, ReUI wins. Never redefine `--ease-out` or any Tailwind built-in.
 
-Skill + MCP are a team: this skill is the workflow (how to find, install, read the API, and adapt by reuse); the MCP is the live data and the hands (search, get_component, install commands). Your job: find the right item, install it with the shadcn CLI, read its real API, and **adapt by reuse** - wire real data and theme it; do not hand-roll or restyle what ReUI already provides. This skill **layers on the shadcn skill**: follow that for generic rules (spacing, `cn()`, semantic colors, forms); follow this for everything ReUI-specific.
+## Before you write UI code, write this plan in your reply
 
-## The core loop (MCP-native)
+1. Surface: operational, marketing or flow.
+2. Archetype: 1 from [compose.md](./rules/compose.md).
+3. Container family: Frame-led, Card-led, or Cards for the KPIs and a Frame for the table.
+4. Density: dense (`size="sm"` on controls that take it) or default.
+5. Regions: top to bottom, each with the item that fills it.
+6. States: empty, loading and error for each data region.
+7. Motion: none, or the 1 entrance.
 
-1. **Find** - call the ReUI MCP `search` tool with the user's intent. It returns a ranked, scored list across components/examples/blocks/icons, each with an `install` command, `previewUrl`, `docsUrl`, and `componentsUsed`. Pass hints (`type`, `component`, `category`, `features`, `free`) when you can infer them.
+Then build exactly that plan; if you deviate, say where.
-2. **Install** - run the returned command non-interactively (`npx shadcn@latest add @reui/<name> --yes`). The CLI resolves deps, aliases, and the base/style from `components.json`. See [cli.md](./rules/cli.md).
-3. **Read the API (on your base)** - first note your base from `components.json` -> `style` (`base-nova` -> Base UI, `radix-nova` -> Radix UI). For each component an item uses, call `get_component(name)` and read its **inline `api`** (no web fetch); then `get_examples(name)` to install a worked example and copy its composition - the installed files are already in your base. Whenever you work with a component's API, also **share its `docsUrl`** (the primitive's API documentation page) with the user so they have the full reference. See [components.md](./rules/components.md).
-4. **Adapt (reuse-first)** - swap demo data for real data, fix icon imports, align tokens. Do not redesign. See [adapting.md](./rules/adapting.md).
 
-**Always show the preview.** Every item a tool returns carries a `previewUrl` (a live preview page). Whenever you list, recommend, or present ReUI items to the user - blocks, components, examples, or icons, whether from `search`, `search_icons`, `list_components`, `compose_page`, or any getter - include each item's `previewUrl` so they can SEE it before installing. Blocks and examples open an individual live preview; icons and components link to their live category/component page. Never present an item without its preview link.
+## The loop
 
-If the ReUI MCP is not configured, fall back to `npx shadcn@latest search @reui -q "..."` then `add` - but the MCP gives scored matches + inline APIs; prefer it.
+1. **Find**: `compose_page` for a full page, else `search` with `type` and `component` hints.
+2. **Install**: `npx shadcn@latest add @reui/<name> --yes`, with `<name>` exactly as a tool returned it (shadcn/ui parts install by bare name: `npx shadcn@latest add button --yes`).
+3. **Read the API on your base**: `components.json` style `base-nova` is Base UI, `radix-nova` is Radix. Call `get_component([...componentsUsed])` once, then install 1 `get_examples` example and read its files.
+4. **Adapt**: change data, copy and props; keep the structure and classes.
+5. **Check**, after every UI edit, follow-ups included: `validate_usage` on any name you did not read, the self-checks C1 to C4 in [quality.md](./rules/quality.md), [anti-patterns.md](./rules/anti-patterns.md), `get_audit_checklist`, typecheck, lint. Hand back in the 5 lines quality.md gives.
 
-## Commands
+Show every item's `previewUrl` whenever you list, recommend or install it.
 
-Run ReUI as explicit slash commands (via the ReUI MCP) **or** just ask in plain language - both run the same workflow.
+## Rules that always apply
 
-| Command     | Invoke                         | Does                                                                                                               |
+- **Owners**: tables `data-grid`, filtering `filters`, boards `kanban`, modals `Dialog` (destructive confirms `AlertDialog`), toasts `sonner`; never hand-roll.
-| ----------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
+- **Color**: semantic tokens for meaning; state colors through Badge or Alert variants (`@reui/badge` and `@reui/alert`, not shadcn's); raw palette only for categorical identity, always with its `dark:` pair.
-| **build**   | `/mcp__reui__build <what>`     | Compose a page/section/feature from ReUI: plan → install → read API → adapt → craft → audit.                       |
+- **Contrast**: never `text-success` or `text-warning` as body text in light mode, and never `text-<state>-foreground` on a solid `bg-<state>` fill.
-| **add**     | `/mcp__reui__add <item>`       | Find & install one component/example/block/icon and wire it in.                                                    |
+- **Shape**: no `rounded-*` except `rounded-full`, and no `shadow-*`, `h-*` or `px-*` on Button, Input, SelectTrigger, Card or Frame; pick `size` and `variant`. A panel is a Card or Frame, never a div with `rounded-lg border`.
-| **fix**     | `/mcp__reui__fix [target]`     | Diagnose & fix ReUI usage: wrong/undocumented props, base/radix mismatch, missing states, a11y/scroll.             |
+- **Spacing**: `gap-*` on the parent, not `space-y-*`; `gap-1` to `gap-2` in a row, `gap-3` to `gap-4` between groups, `gap-6` to `gap-8` between sections; title to description `gap-0.5`.
-| **improve** | `/mcp__reui__improve [target]` | Refine + extend existing ReUI UI to a production-exceptional bar (hierarchy, density, states, responsive, motion). |
+- **Type**: `text-sm` body, `text-xs` meta (controls keep their style's size), `font-medium` or `font-semibold`, headings `tracking-tight`, numbers `tabular-nums`; nothing below `text-xs` except illustrative mini-UI.
+- **Motion**: installed components own their motion. Your own code uses `transition-colors` for hover and press, `duration-200 ease-[cubic-bezier(0.22,1,0.36,1)]` to reveal, `duration-300` with the same curve at most (media hover zoom: 500), and `SPRING` (`{ type: "spring", visualDuration: 0.25, bounce: 0 }`) for drag and reorder. At most 1 entrance animation per view (a stagger counts as 1). Animate only `transform` and `opacity`, CSS before motion/react, never set state per frame; `<MotionConfig reducedMotion="user">` once at the root plus `motion-reduce:` on CSS, where reduced motion means a calm fade, not nothing.
+- **Surfaces**: prefer 1 container family per screen. With a KPI row and a table, use FramePanels for both, or Cards for the KPIs and a Frame for the table; never a Card inside a Frame or a Frame inside a Card. Icons come from 1 library, the `iconLibrary` in `components.json` (on Ultimate, 1 Motion Icon on the primary action is the exception).
+- **States**: each data region has empty (1 action), loading (a `Skeleton` in the real layout) and error (Retry) states; light and dark work.
+- **Never invent**: never write a prop, variant or `@reui/` name you did not read in an `api`, an installed file or a tool result.
 
-Invocation differs slightly per agent (`/mcp__reui__build` in Claude Code/Cursor/Windsurf, `/mcp.reui.build` in VS Code). No command surface? Just describe what you want - this skill drives the identical loop.
+## Topics
 
-## When to reach for ReUI vs plain shadcn
+Always read `compose` before building a screen, and `quality` plus `anti-patterns` at the Check step. Besides those, read only the topic files the task touches, at most 4 (for example `color`, `motion`, `layout` and `components`).
 
-| Need                                                                 | Reach for                                                                 |
+Installed skill: open the file. MCP only: call `get_agent_skill` with the topic, or fetch `https://mcp.reui.io/skill/reui/<file>`.
-| -------------------------------------------------------------------- | ------------------------------------------------------------------------- |
-| A full page or section (dashboard, billing, auth, pricing, settings) | `compose_page` first (plans sections + best blocks), then ReUI **blocks** |
-| A data table with sorting/filtering/pagination/virtualization        | the **data-grid** component (never hand-roll a `<table>`)                 |
-| A drag-and-drop board                                                | the **kanban** component                                                  |
-| Advanced column filtering, date range, tree, stepper, ...            | the matching ReUI **component**                                           |
-| A single generic control already in shadcn (Button, Dialog, Select)  | plain **shadcn**                                                          |
 
-## Detailed references
+| Topic           | File                                               | Read when                                     |
+| --------------- | -------------------------------------------------- | --------------------------------------------- |
+| `compose`       | [rules/compose.md](./rules/compose.md)             | before building any screen or section         |
+| `workflow`      | [rules/workflow.md](./rules/workflow.md)           | `compose_page`, search hints, installing      |
+| `adapting`      | [rules/adapting.md](./rules/adapting.md)           | changing an installed block or example        |
+| `components`    | [rules/components.md](./rules/components.md)       | `data-grid`, `filters`, `kanban` and the rest |
+| `color`         | [rules/color.md](./rules/color.md)                 | any color, status, chart or dark mode choice  |
+| `theming`       | [rules/theming.md](./rules/theming.md)             | the styles, radius, density and sizes         |
+| `motion`        | [rules/motion.md](./rules/motion.md)               | any transition, animation or motion/react     |
+| `layout`        | [rules/layout.md](./rules/layout.md)               | house defaults, hierarchy, surfaces, states   |
+| `icons`         | [rules/icons.md](./rules/icons.md)                 | icon imports, Motion Icons                    |
+| `quality`       | [rules/quality.md](./rules/quality.md)             | the done gate: self-checks, a11y, hand-back   |
+| `anti-patterns` | [rules/anti-patterns.md](./rules/anti-patterns.md) | the Check step: drift pairs, stale habits     |
+| `registry`      | [rules/registry.md](./rules/registry.md)           | free vs Pro vs Ultimate, base vs radix        |
+| `cli`           | [rules/cli.md](./rules/cli.md)                     | `components.json`, license key, install flags |
+| `tools`         | [tools.md](./tools.md)                             | MCP result shapes, errors, quota, commands    |
 
-- [rules/registry.md](./rules/registry.md) - the four types, the @reui registry, base/radix, free vs premium + license
+## Plans
-- [rules/workflow.md](./rules/workflow.md) - the find -> install -> read-API -> adapt loop (most important)
+
-- [rules/components.md](./rules/components.md) - the 22 components, the data-grid contract, base vs radix
+Free covers components, `c-*` examples, this skill and the MCP (a ReUI account and a daily allowance). Pro adds blocks. Ultimate adds Motion Icons. A plan miss is a normal answer with `locked: true`; keep building with what the plan covers.
-- [rules/adapting.md](./rules/adapting.md) - reuse-first: preserve the design (no over-customizing), reuse examples + a block's own elements, real data, don't invent APIs
-- [rules/craft.md](./rules/craft.md) - make it exceptional: point of view, hierarchy, density, states, responsive, motion, the bar
-- [rules/quality.md](./rules/quality.md) - security, accessibility, and scroll gates (the done gate)
-- [rules/styling.md](./rules/styling.md) - ReUI extended tokens, theme adaptation, density
-- [rules/icons.md](./rules/icons.md) - portable icons, swapping imports, Motion Icons (static + animated)
-- [tools.md](./tools.md) - the ReUI MCP: golden path, the 19 tools, token rules, result shapes, errors

.agents/skills/reui/rules/adapting.md

@@ -30,14 +30,11 @@ The design IS the product. A ReUI block/component encodes senior-designer decisi
 
 - Replace every placeholder with the user's real data. Model it as **typed data structures** and **map over arrays** - never duplicate JSX per row/card. Keep small block-specific formatters next to the data.
 - Wire the real source (columns, fields, fetch). For `data-grid`, implement the server fetch contract if the user needs server-side data.
+- Never read `window`, `localStorage`, `location`, `Date.now()` or `Math.random()` in render or in a `useState` initializer; read them in an effect.
 - **Type from the component API, derive during render.** Type domain state through the component's own types - e.g. map status to `BadgeProps["variant"]` via a typed `Record<Status, …>` - instead of stringly-typed values. Compute view state during render; don't mirror derived data into `useState`/`useEffect`.
 - **Adapt on the right base.** Use the API for the project's base (Base UI vs Radix - see [components.md](./components.md)); the installed files are already base-correct, so reuse their shape rather than translating from memory.
 
 ## Believable content (no AI tells)
 
 - Use realistic labels, counts, timestamps, and statuses that map to a real workflow.
-- No decorative buttons, fake tabs, meaningless toggles, equal-weight card walls, empty gradients, ornamental icons, or generic SaaS filler. Every element should do something.
+- No decorative buttons, fake tabs, meaningless toggles, equal-weight card walls, empty gradients, ornamental icons, or generic SaaS filler. Every element should do something, and the layout must still hold with long names, empty values and crowded data.
-
-## Operational surfaces (settings / profile / admin)
-
-Pick ONE archetype and keep the family consistent: a vertical rail (3-6 sections), horizontal tabs (5-8), or a frame/stack. Prefer `frame` for tool-like surfaces, a card for profile-like ones. Don't mix archetypes in one surface.

.agents/skills/reui/rules/anti-patterns.md

@@ -0,0 +1,117 @@
+# Anti-patterns
+
+Read this at the Check step, after the self-checks in [quality.md](./quality.md). Each entry: an `Incorrect:` line, then its `Correct:` fix. Detector: the self-check that prints it (C1 to C4), or `visual` when reading finds it. Source: the pilot run that wrote it, or `default` (a training-data habit).
+
+## Pairs
+
+### 1. Stagger on every row (Detector: C2; source: run-08)
+
+Incorrect: `transition={{ delay: i * 0.05 }}` on all 50 rows of a table
+Correct: 1 stagger of at most 6 items, `staggerChildren: 0.05`, never on table rows
+
+### 2. Transition everything on a card (Detector: C1; source: runs 10, 15)
+
+Incorrect: `className="rounded-xl border transition-all duration-300 hover:shadow-lg"`
+Correct: `transition-colors` for a hover color; a lift is `transform` over a static shadow
+
+### 3. Raw palette for a state (Detector: C3; source: runs 08, 15)
+
+Incorrect: `<span className="text-green-600">Active</span>`
+Correct: `<Badge variant="success-light">Active</Badge>`
+
+### 4. Animated table rows (Detector: C1; source: runs 01, 08, 17)
+
+Incorrect: `<motion.tr whileHover={{ backgroundColor: "var(--muted)" }}>`
+Correct: `data-grid` rows, which already hover; never animate rows with motion/react
+
+### 5. Hover scale on the default spring (Detector: C1; source: runs 01, 10)
+
+Incorrect: `<motion.div whileHover={{ scale: 1.03 }}>` with no `transition`
+Correct: CSS first: `transition-colors` for hover, `transition-transform active:scale-[0.98] motion-reduce:transition-none` for press
+
+### 6. Entrance from scale 0 (Detector: C1; source: runs 10, 15)
+
+Incorrect: `<motion.div initial={{ scale: 0 }} animate={{ scale: 1 }}>`
+Correct: `initial={{ opacity: 0, scale: 0.95 }}`: start at scale 0.95 or more and move 8px or less
+
+### 7. Hand-built modal or toast (Detector: C4; source: run-15)
+
+Incorrect: `<div className="fixed inset-0 z-50 bg-black/50">` as a modal
+Correct: `Dialog`, `AlertDialog` for a destructive confirm, `sonner` for toasts; a palette: `<CommandDialog><Command><CommandInput /><CommandList>`, as `CommandDialog` holds no `Command`
+
+### 8. Native select (Detector: C4; source: run-15)
+
+Incorrect: `<select>` with `<option>` children
+Correct: the shadcn `Select`, or `NativeSelect` when the native control is wanted
+
+### 9. Hand-rolled table (Detector: C4; source: run-17)
+
+Incorrect: `<table>` with mapped `<tr>` rows
+Correct: `data-grid`: a `useTable({ features: dataGridFeatures })` instance passed as `table={table}`, columns typed `ColumnDef<DataGridFeatures, Order>[]`
+
+### 10. Button inside a clickable wrapper (Detector: visual; source: run-02)
+
+Incorrect: `<motion.button><Button>Save</Button></motion.button>`
+Correct: the `Button` alone: 1 interactive element, and it already has a press state
+
+### 11. Radix asChild on Base UI (Detector: C4; source: run-17, judge)
+
+Incorrect: `<DialogTrigger asChild><Button>Open</Button></DialogTrigger>`
+Correct: `<DialogTrigger render={<Button variant="outline" />}>Open</DialogTrigger>` on base-nova; a link: `<Button nativeButton={false} render={<Link href="/x" />}>`
+
+### 12. Component defined inside render (Detector: visual; source: Haiku runs, judge)
+
+Incorrect: `const Row = () => <div />` inside the page component
+Correct: declare `Row` at module scope and pass it props, so React keeps its state
+
+### 13. A panel inside a panel (Detector: visual; source: run-05)
+
+Incorrect: `<FramePanel><Card>...</Card></FramePanel>`, or a Card inside a Card
+Correct: FramePanels for both, or Cards for the KPIs and a Frame for the table; never a Card inside a Frame
+
+### 14. Small badges (Detector: visual; source: runs 03, 04, 06, 07, 10, 13, 16, 18)
+
+Incorrect: `<Badge variant="success-light" size="sm">Paid</Badge>`
+Correct: `<Badge variant="success-light">Paid</Badge>` with `size` left off
+
+### 15. A colored circle behind every KPI icon (Detector: visual; source: default)
+
+Incorrect: `<div className="rounded-full bg-blue-100 p-2">` around each KPI icon
+Correct: the number leads; at most 1 `text-muted-foreground` icon
+
+### 16. Gradient text (Detector: visual; source: default)
+
+Incorrect: `bg-linear-to-r from-violet-500 to-pink-500 bg-clip-text text-transparent` on a heading
+Correct: `text-foreground` and `tracking-tight`; color carries meaning, not decoration
+
+### 17. A dot or check for the current item (Detector: visual; source: default)
+
+Incorrect: a dot or a `Check` icon beside the active nav item
+Correct: `bg-muted` (`dark:bg-muted/50`) plus `aria-current`
+
+### 18. framer-motion (Detector: C4; source: default)
+
+Incorrect: `import { motion } from "framer-motion"`
+Correct: `import { motion } from "motion/react"`
+
+### 19. Clock or randomness in render (Detector: C4; source: default, near run-05)
+
+Incorrect: `const [id] = useState(() => Math.random())`
+Correct: `useId()` for ids; read the clock in an effect or an event handler
+
+### 20. Placeholder data (Detector: visual; source: default)
+
+Incorrect: "John Doe", "Acme Inc" and round figures like `$1,000`
+Correct: specific, plausible data: "Mara Lind", `$1,284.50`, 37 open orders
+
+## Your training data is out of date
+
+| Old habit                     | In this stack                                                   |
+| ----------------------------- | --------------------------------------------------------------- |
+| `framer-motion`               | `motion/react`                                                  |
+| `<DataGrid data columns>`     | `useTable({ features: dataGridFeatures })` plus `table={table}` |
+| A fixed div modal             | `Dialog` or `AlertDialog`                                       |
+| A hand-built toast            | `sonner`                                                        |
+| `w-5 h-5`                     | `size-5`                                                        |
+| never `space-y-4` in new code | `flex flex-col gap-4`                                           |
+| `asChild`                     | `render` on base-nova                                           |

.agents/skills/reui/rules/cli.md

@@ -2,13 +2,13 @@
 
 ## Registry setup (one-time, per project)
 
-Free items (the 22 components and all `c-*` examples) need only the plain string registry in `components.json`:
+Free items (the ReUI components and all `c-*` examples) need only the plain string registry in `components.json`:
 
 ```json
 { "registries": { "@reui": "https://reui.io/r/{style}/{name}.json" } }
 ```
 
-Premium items (blocks; Motion Icons and templates) require a ReUI license at install:
+Premium items (blocks and Motion Icons) require a ReUI license at install:
 
 1. Add the key to `.env.local`:
 
@@ -52,6 +52,13 @@ bunx --bun shadcn@latest add @reui/<name> --yes # bun
 
 `--yes` skips confirmation prompts. The CLI auto-detects the package manager from the lockfile (there is no `--package-manager` flag). It also resolves the correct base+style variant from `components.json`, so do not pass a style.
 
+`<name>` is the item name exactly as a ReUI tool returned it. The `@reui/` prefix is for ReUI items only:
+
+- The shadcn/ui components ReUI builds on (`button`, `dialog`, `select`, `card`, ...) are not ReUI items. Install them from shadcn by bare name: `npx shadcn@latest add button --yes`.
+- `badge` and `alert` exist in both registries: `@reui/badge` and `@reui/alert` are ReUI's components, bare `badge` and `alert` are shadcn's.
+- In an item's `registryDependencies`, a `@reui/<name>` entry is a ReUI item and a bare name is a shadcn/ui item. `add` installs both, so you never add them by hand.
+- Templates are not registry items. Download a template from its page on reui.io (https://reui.io/templates?ref=skill); no `shadcn add` command installs one.
+
 ## Handling prompts and conflicts
 
 - **Always pass `--yes`** so the CLI does not block on confirmation prompts.
@@ -61,7 +68,10 @@ bunx --bun shadcn@latest add @reui/<name> --yes # bun
 
 ## Free vs premium boundary
 
-- Public, no key: `c-*` examples and the 22 components (`@reui/data-grid`, `@reui/badge`, ...) that those examples depend on.
+- Public, no key: `c-*` examples and the ReUI components (`@reui/data-grid`, `@reui/badge`, ...) that those examples depend on.
-- Key required at install: blocks (`@reui/<category>-N`) need a Pro or Ultimate license; Motion Icons (`@reui/icons/...`) and templates need Ultimate.
+- Key required at install: blocks (`@reui/<category>-N`) need a Pro or Ultimate license; Motion Icons (`@reui/icons/...`) need Ultimate.
+- Templates are downloads from their page on reui.io, not installs: free ones need only a ReUI account, and each template's page says what the others need.
 
-If an install 401/403s, the license key is missing, invalid, or the plan does not cover that resource (blocks: Pro or higher; icons and templates: Ultimate). Point the user to https://reui.io/account?ref=skill (their key) or https://reui.io/pricing?ref=skill (upgrade).
+If an install 401/403s, the license key is missing, invalid, or the plan does not cover that resource (blocks: Pro or higher; icons: Ultimate). Point the user to https://reui.io/account?ref=skill (their key) or https://reui.io/pricing?ref=skill (upgrade).
+
+If an install 404s, the name is not a ReUI item. Read the CLI's error message, which names the right install command when there is one, or look the name up with `get_install_command` or `search`.

.agents/skills/reui/rules/color.md

@@ -0,0 +1,74 @@
+# Color
+
+Semantic tokens carry meaning; raw palette carries identity. Check every color in light and dark.
+
+## Tokens
+
+Surfaces, text and borders use your theme tokens (`background`, `foreground`, `card`, `muted`, `muted-foreground`, `primary`, `accent`, `destructive`, `border`, `ring`), so a theme change re-tints them. ReUI adds state tokens: `--success`, `--info` and `--warning` with their `-foreground`, `--destructive-foreground` (beside shadcn's `--destructive`) and `--invert` with `--invert-foreground` for inverted surfaces.
+
+They are not part of shadcn's base theme. Installing `@reui/badge`, `@reui/alert` or `@reui/code-block` adds them to your global CSS; until one of those is installed, classes such as `bg-success` generate nothing. Check for `--success:` in your global CSS. If it is missing, add these values to `:root` and `.dark`, plus a `--color-<name>: var(--<name>)` line for each under `@theme inline`:
+
+<!-- gen:tokens:start -->
+
+| Token                      | `:root`                    | `.dark`                    |
+| -------------------------- | -------------------------- | -------------------------- |
+| `--destructive-foreground` | `var(--color-red-800)`     | `var(--color-red-600)`     |
+| `--success`                | `var(--color-emerald-500)` | `var(--color-emerald-500)` |
+| `--success-foreground`     | `var(--color-emerald-900)` | `var(--color-emerald-600)` |
+| `--info`                   | `var(--color-violet-500)`  | `var(--color-violet-500)`  |
+| `--info-foreground`        | `var(--color-violet-900)`  | `var(--color-violet-600)`  |
+| `--warning`                | `var(--color-yellow-500)`  | `var(--color-yellow-500)`  |
+| `--warning-foreground`     | `var(--color-yellow-900)`  | `var(--color-yellow-600)`  |
+| `--invert`                 | `var(--color-zinc-900)`    | `var(--color-zinc-700)`    |
+| `--invert-foreground`      | `var(--color-zinc-50)`     | `var(--color-zinc-50)`     |
+
+<!-- gen:tokens:end -->
+
+## Meaning vs identity
+
+- Meaning (a state, emphasis, a surface, text) takes semantic tokens, through the Badge or Alert variant that already carries them.
+- Identity (a chart series, a tag color, a team, an avatar) may use raw palette: ReUI's own lint keeps it on purpose. Always hand-pair it for dark, as in `text-violet-600 dark:text-violet-400`.
+- **Incorrect:** `<span className="text-green-600">Active</span>` (raw palette for a state). **Correct:** `<Badge variant="success-light">Active</Badge>`.
+
+## State recipes
+
+- State badge: `<Badge variant="success-light">` from `@/components/reui/badge` (`@reui/badge`; shadcn's `components/ui/badge` has no `-light` variants or `size`), also `info-light`, `warning-light`, `destructive-light`, and `primary-light` for brand or pending. Neutral badge: `variant="outline"`. A solid `<Badge variant="success">` (white text) only when the user asks for a solid fill.
+- Inline status text: `text-success-foreground dark:text-success`, the pair the ReUI Badge uses.
+- Tinted surface: `border-success/15 bg-success/10 text-success-foreground dark:border-success/25 dark:bg-success/15 dark:text-success` (the Badge's `success-light` recipe).
+- A message block: `<Alert variant="success">` from `@/components/reui/alert` (`@reui/alert`), also `info`, `warning` and `destructive`.
+- Never `text-success` or `text-warning` as body text in light mode, and never `text-<state>-foreground` on a solid `bg-<state>` fill.
+
+## Contrast
+
+<!-- gen:contrast:start -->
+
+Light is `text-<tone>-foreground` on `background` and on the `bg-<tone>/10` tint; dark is `dark:text-<tone>` on `background` and on the `dark:bg-<tone>/15` tint. Ratios against the neutral theme, rounded down; body text needs 4.5:1.
+
+| Tone          | Light | Light on tint | Dark | Dark on tint |
+| ------------- | ----- | ------------- | ---- | ------------ |
+| `success`     | 9.6   | 8.8           | 8.0  | 6.6          |
+| `warning`     | 8.6   | 8.1           | 10.3 | 8.1          |
+| `info`        | 11.0  | 9.7           | 4.4  | 3.9          |
+| `destructive` | 8.3   | 7.0           | 6.8  | 5.7          |
+
+Raw state color as light-mode text on `background`: `text-success` 2.4, `text-warning` 1.9, `text-info` 4.4.
+
+<!-- gen:contrast:end -->
+
+Info text in dark mode is under 4.5:1: where it must be read, write the words in `text-foreground` and color only the icon.
+
+## Dark mode
+
+- shadcn uses the class strategy: `@custom-variant dark (&:is(.dark *));` in your global CSS, and `.dark` on `<html>` turns it on.
+- Tokens switch by themselves; raw palette needs its `dark:` pair.
+- Check both themes before you hand back.
+
+## Charts
+
+Check your theme's `--chart-1` to `--chart-5` in both modes. The current shadcn themes keep the same chart colors in light and dark; some themes (reui.io's own among them) change hue. For a series whose identity must hold in both modes, use a palette var such as `var(--color-violet-500)`, as ReUI blocks do.
+
+<!-- gen:charts:start -->
+
+Same chart colors in light and dark: neutral, stone, zinc, mauve, olive, mist, taupe, amber, blue, cyan, emerald, fuchsia, green, indigo, lime, orange, pink, purple, red, rose, sky, teal, violet, yellow. Changes hue between light and dark: reui.io's own theme.
+
+<!-- gen:charts:end -->

.agents/skills/reui/rules/components.md

@@ -1,35 +1,36 @@
 # ReUI components
 
-The 22 ReUI building blocks: `alert`, `autocomplete`, `badge`, `cascader`, `code-block`, `data-grid`, `date-selector`, `event-calendar`, `filters`, `frame`, `gantt`, `icon-stack`, `icon-tile`, `kanban`, `number-field`, `phone-input`, `rating`, `scrollspy`, `sortable`, `stepper`, `timeline`, `tree`. Examples and blocks are composed from these.
+The ReUI components: `alert`, `autocomplete`, `badge`, `cascader`, `code-block`, `color-picker`, `data-grid`, `date-selector`, `emoji-picker`, `event-calendar`, `file-type`, `filters`, `frame`, `gantt`, `icon-stack`, `icon-tile`, `kanban`, `number-field`, `phone-input`, `rating`, `scrollspy`, `signature-pad`, `sortable`, `stepper`, `time-picker`, `timeline`, `tree`. Examples and blocks are composed from these. `file-upload` is a ReUI component too, shipped as the `use-file-upload` hook rather than a primitive, so it is not in that list: install it as `@reui/use-file-upload`, which writes `hooks/use-file-upload.ts` (import `useFileUpload` from `@/hooks/use-file-upload`), call `get_component("file-upload")` for its API and `get_examples("file-upload")` for its free examples. There is no `@reui/file-upload`: the UI is a `c-file-upload-<n>` example, and one built on the hook installs it.
 
-**Rule one: never guess a component's API. Read it first.** Call **`get_component(name)`** for its inline `api` (props + usage, no web fetch), and **share the result's `docsUrl`** (the component's API documentation page) with the user whenever you work with that component's API, so they have the full reference (the `/llms.txt` index is a further fallback). Then call **`get_examples(name)`** to install a worked example and copy real composition. The contracts below are first-try orientation (required props, composition shape, the one gotcha); the inline `api` is the deeper reference. Do not assume you received all of it: a very large API (`cascader`, `filters`, `data-grid`) is trimmed on heading boundaries to fit your context, and every dropped heading is named in `sectionsOmitted`, so a trimmed capsule is a partial read - follow the response's own `next` hint to pull back the part you need instead of guessing at it. `validate_usage` always checks the FULL API, trimmed capsule or not. No single block fits? Compose: search the components you need, read each `get_component`, install a `get_examples` example per component, and adapt.
+**Rule one: never guess a component's API.** Call **`get_component(name)`** for its inline `api` and share its `docsUrl` with the user, then **`get_examples(name)`** and copy a worked example. The contracts below are first-try orientation (required props, shape, the one gotcha); the `api` is the reference. A large `api` (`cascader`, `filters`, `data-grid`) is trimmed on heading boundaries: `sectionsOmitted` names what was dropped and the response's `next` hint pulls it back. `validate_usage` always checks the full API. No single block fits? Compose from components, 1 `get_examples` example each ([compose.md](./compose.md)).
 
 ## data-grid (the flagship - read its API every time)
 
 `data-grid` wraps TanStack Table v9. It is NOT a styled `<table>` and does NOT take `data`/`columns` props directly. The contract:
 
-- Build a TanStack table instance with `useTable({ features: dataGridFeatures, ... })` (columns, data). `dataGridFeatures` is exported by the primitive and already bundles sorting, filtering, pagination, row selection, expanding, pinning, resizing and faceting, so there are no per-table row models to wire.
+- Build a TanStack table with `useTable({ features: dataGridFeatures, ... })`. `dataGridFeatures` already bundles sorting, filtering, pagination, selection, expanding, pinning, resizing and faceting: no row models to wire.
 - Pass that instance to `<DataGrid table={table} recordCount={total}>`.
 - Compose the body with `DataGridTable` inside `DataGrid`, and enable features through `tableLayout` (e.g. `{ headerSticky: true, columnsResizable: true }`), not ad-hoc classes.
-- Server-side data uses the documented fetch shape (`recordCount` is the total for pagination).
+- Server-side data: `manualPagination: true` and `rowCount: total` in `useTable`, `recordCount={total}`, and fetch with the table's `pagination` state (`DataGridPagination` changes `pageSize`).
+- Columns are TanStack v9, `ColumnDef<DataGridFeatures, Order>[]`, never v8's `ColumnDef<Order>[]`; a sortable column's `header` renders `DataGridColumnHeader` (a string header cannot sort).
+- Imports, each from `@/components/reui/data-grid/<file>`: `DataGrid`, `DataGridFeatures` and `dataGridFeatures` from `data-grid`, `DataGridColumnHeader` from `data-grid-column-header`, `DataGridTable` from `data-grid-table`, `DataGridPagination` from `data-grid-pagination`, `DataGridScrollArea` from `data-grid-scroll-area`; `useTable` and `ColumnDef` from `@tanstack/react-table`.
+- States: `isLoading` renders skeleton rows from each column's `meta.skeleton` (a `ReactNode`: `<Skeleton className="h-4 w-20" />`); `emptyMessage` (a `ReactNode`) fills a grid with no rows: after a failed fetch, pass it the error with a Retry.
+- Numeric column: `meta: { headerClassName: "text-end [&>div]:justify-end", cellClassName: "text-end tabular-nums" }`.
+- Rows edited in place: `autoResetPageIndex: false` in `useTable`, or each write jumps back to page 1.
+- In a Frame: `<FramePanel className="p-0">` around `DataGridScrollArea` and `DataGridTable`, plus `tableClassNames={{ base: "min-w-[720px]" }}` so on a phone the table scrolls inside the panel instead of squeezing.
 
 ```tsx
-const table = useTable({
+const columns = useMemo<ColumnDef<DataGridFeatures, Order>[]>(() => [
-  features: dataGridFeatures,
+  { accessorKey: "total", header: ({ column }) => <DataGridColumnHeader title="Total" column={column} /> },
-  data,
+], [])
-  columns,
+const table = useTable({ features: dataGridFeatures, data, columns })
-})
 
 <DataGrid table={table} recordCount={data.length}>
   <DataGridTable />
 </DataGrid>
 ```
 
-Common mistakes:
+Never a raw `<table>` or hand-rolled pagination: the API covers sticky header, virtualization and row selection.
-
-- **Incorrect:** `<DataGrid data={rows} columns={cols} />` - these props do not exist. **Correct:** build a `useTable({ features: dataGridFeatures, ... })` instance and pass `table={table}` + `recordCount`.
-- **Incorrect:** a raw `<table>` / hand-rolled pagination. **Correct:** use `data-grid`; read its API for sticky header, pagination, virtualization, row selection.
-- **Incorrect:** styling rows/cells with arbitrary classes. **Correct:** drive layout via `tableLayout` and the primitive's `DataGridColumnMeta` (e.g. `cellClassName`, `headerTitle`), set through the bundle's `columnMeta` slot.
 
 ## event-calendar
 
@@ -69,7 +70,10 @@ Common mistakes:
   <KanbanBoard>
     {Object.entries(cols).map(([id, items]) => (
       <KanbanColumn key={id} value={id}>
-        <KanbanColumnHandle><h3>{id}</h3></KanbanColumnHandle>
+        <div className="flex items-center justify-between">
+          <h3>{id}</h3>
+          <KanbanColumnHandle aria-label="Move column"><GripVertical /></KanbanColumnHandle>
+        </div>
         <KanbanColumnContent value={id}>
           {items.map((i) => (
             <KanbanItem key={i.id} value={i.id}>
@@ -84,7 +88,7 @@ Common mistakes:
 </Kanban>
 ```
 
-**Gotcha:** state is `Record<columnId, T[]>`. Each `KanbanColumnContent value` must match its parent `KanbanColumn value`. Omit `KanbanOverlay` and the drag preview silently breaks.
+**Gotcha:** state is `Record<columnId, T[]>`. Each `KanbanColumnContent value` must match its parent `KanbanColumn value`. Omit `KanbanOverlay` and the drag preview silently breaks. `KanbanColumnHandle` is hidden until its column is hovered or it has keyboard focus, so keep the column title outside it.
 
 ## sortable
 
@@ -127,9 +131,9 @@ const [query, setQuery] = useState<FilterQuery>(() => createFilterQuery())
 <Filters fields={fields} query={query} onQueryChange={setQuery} />
 ```
 
-**Gotcha:** the state is a TREE, not a list of chips. `FilterQuery` is a group of rules joined by `and`/`or` and a group may hold another group, so `(A and B) or C` is expressible; a rule is `{ id, type: "rule", path: ["status"], operator, value }` and `path` is the whole nested attribute path, root first. The pre-rewrite API is GONE: there is no `filters`/`onChange` prop, no `FilterFieldConfig` (fields are `FilterField`, nested through their own `fields`, keyed `id` not `key`), and no `createFilter()` - it minted ids inside a pure function and broke hydration, so ids now come from `createFilterIdFactory(seed)` seeded off `useId`, and `createFilterQuery()` / `createFilterRule()` take one. Read the query back with `flattenFilterConditions` (`{ path, field, operator, values, negated }` per rule, incomplete rules skipped) and walk the tree yourself when the parentheses carry meaning - the primitive compiles nothing, no SQL, no query string.
+**Gotcha:** the state is a TREE, not a list of chips. `FilterQuery` is a group of rules joined by `and`/`or` and a group may hold another group, so `(A and B) or C` is expressible; a rule is `{ id, type: "rule", path: ["status"], operator, value }` and `path` is the whole nested attribute path, root first. The pre-rewrite API is GONE: there is no `filters`/`onChange` prop, no `FilterFieldConfig` (fields are `FilterField`, nested through their own `fields`, keyed `id` not `key`), and no `createFilter()`: ids come from `createFilterIdFactory(seed)` seeded off `useId` (never minted in render), and `createFilterRule({ id })` takes one, `createFilterQuery()` none. Read the query back with `flattenFilterConditions` (`{ path, field, operator, values, negated }` per rule, incomplete rules skipped) and walk the tree yourself when the parentheses carry meaning - the primitive compiles nothing, no SQL, no query string.
 
-`variant` picks the chrome over that one query: `"basic"`, the default, is the flat chip row for a toolbar over a table; `"advanced"` is the condition builder, hung off a trigger or rendered in place with `advancedMode="inline"`. Both read and write the same tree, so a saved view built in one opens in the other. Other props worth knowing before you hand-roll them: `size` is two rungs, `"sm" | "default"`, resolved per style (there is no `lg`); `reorderable` turns on drag and Alt+Arrow row moves in the builder; `onBeforeQueryChange` is the ONE veto point for every write (return `false` to refuse, it cannot rewrite); `editors` registers custom value editors a field selects by `editor` name; `labels` / `operatorLabels` own every rendered string; `pathCollapse` + `maxPathSegments` shorten deep attribute paths; `renderChip` / `renderValue` / `renderEmpty` replace rendered parts. On a field, `loadOptions` supplies async options with paging and `resolveValues` renders a chip restored from a saved view whose option was never loaded. Pairs naturally with `data-grid`.
+`variant` picks the chrome over that one query: `"basic"`, the default, is the flat chip row for a toolbar over a table; `"advanced"` is the condition builder, hung off a trigger or rendered in place with `advancedMode="inline"`. Both read and write the same tree, so a saved view built in one opens in the other. Other props worth knowing before you hand-roll them: `size` is two rungs, `"sm" | "default"`, resolved per style (there is no `lg`); `reorderable` turns on drag and Alt+Arrow row moves in the builder; `onBeforeQueryChange` is the ONE veto point for every write (return `false` to refuse, it cannot rewrite); `editors` registers custom value editors a field selects by `editor` name; `labels` / `operatorLabels` own every rendered string, and `renderChip` / `renderValue` / `renderEmpty` replace rendered parts. On a field, `loadOptions` supplies async options with paging and `resolveValues` renders a chip restored from a saved view whose option was never loaded.
 
 ## cascader
 
@@ -157,9 +161,9 @@ const [query, setQuery] = useState<FilterQuery>(() => createFilterQuery())
 </Cascader>
 ```
 
-**Gotcha:** pressing a branch NAVIGATES, it does not select - only leaves are selectable until you pass `selectable="any"` or a predicate, and once a branch is selectable its chevron becomes the only way to open it. `CascaderInput` must stay inside `CascaderContent` (Base UI refills the query from the selection when the input sits outside the popup). Always include `CascaderStatus`: it is the live region announcing level changes, which the visual breadcrumb does not provide to screen readers. Accepts a flat adjacency list via `getParent` as well as nested `children`. `searchScope="deep"` searches every level and annotates results with their path; `multiple` gives checkbox rows; `inline` + a bare `CascaderPanel` embeds it with no popover.
+**Gotcha:** pressing a branch NAVIGATES, it does not select - only leaves are selectable until you pass `selectable="any"` or a predicate, and once a branch is selectable its chevron becomes the only way to open it. `CascaderInput` must stay inside `CascaderContent` (Base UI refills the query from the selection when the input sits outside the popup). Always include `CascaderStatus`: it is the live region announcing level changes, which the visual breadcrumb does not provide to screen readers. Accepts a flat adjacency list via `getParent` as well as nested `children`. `searchScope="deep"` searches the current level and everything under it, `searchScope="global"` the whole tree from any level, and both annotate results with their path; `multiple` gives checkbox rows; `inline` + a bare `CascaderPanel` embeds it with no popover.
 
-The shape above is `mode="drill"`, the default. `mode="tree"` keeps the same parts (drop `CascaderBreadcrumb`, pass `showBack={false}`, drive expansion with `expanded`/`onExpandedChange`); `mode="columns"` REPLACES `CascaderList` + `CascaderItems` with a single `CascaderColumns`, and has no breadcrumb. Other props worth knowing before you hand-roll them: `cascade` (multi-select only, parent/child selection with indeterminate branches - pair it with `selectable="any"`, since a leaf-only tree can never cascade), `indicator={false}` to drop the single-select check and its gutter (visual only, no-op with `multiple`), `virtualize`/`virtualizeThreshold` plus `CascaderVirtualItems` for long levels, and `getChildren` for async levels with cursor paging, retry on failure and optional `prefetch`. `CascaderFooter` pins commands below the list (`actions` is the quick path) and `CascaderSubmenu` opens one as a side-anchored flyout with the full menu keyboard model. To head a run of rows use `CascaderGroup` wrapping a `CascaderLabel` - a bare label inside a listbox names nothing and is dropped from the accessibility tree - and `CascaderSeparator` for the rule between runs. Every rendered string comes from `labels`, and the panel is RTL-correct under a `DirectionProvider` or `dir="rtl"`.
+The shape above is `mode="drill"`, the default. `mode="tree"` keeps the same parts (drop `CascaderBreadcrumb`, pass `showBack={false}`, drive expansion with `expanded`/`onExpandedChange`); `mode="columns"` REPLACES `CascaderList` + `CascaderItems` with a single `CascaderColumns`, and has no breadcrumb. Other props worth knowing before you hand-roll them: `cascade` (multi-select only, parent/child selection with indeterminate branches - pair it with `selectable="any"`, since a leaf-only tree can never cascade), `virtualize` plus `CascaderVirtualItems` for long levels, and `getChildren` for async levels (paging, retry, `prefetch`). `CascaderFooter` pins commands below the list and `CascaderSubmenu` opens one as a side flyout. To head a run of rows use `CascaderGroup` wrapping a `CascaderLabel` - a bare label inside a listbox names nothing and is dropped from the accessibility tree - and `CascaderSeparator` for the rule between runs.
 
 ## date-selector
 
@@ -285,7 +289,7 @@ const [value, setValue] = useState<DateSelectorValue | undefined>()
 </NumberField>
 ```
 
-**Gotcha:** import from `@/components/ui/number-field`. The accessible label goes on `NumberFieldScrubArea`, not `NumberField`. External API: https://base-ui.com/react/components/number-field
+**Gotcha:** import from `@/components/reui/number-field`. The accessible label goes on `NumberFieldScrubArea`, not `NumberField`. External API: https://base-ui.com/react/components/number-field
 
 ## rating
 
@@ -314,7 +318,7 @@ const [value, setValue] = useState<DateSelectorValue | undefined>()
 </div>
 ```
 
-**Gotcha:** each link's `data-scrollspy-anchor` must match a section `id`. `targetRef` is the scrollable container (defaults to the window).
+**Gotcha:** each link's `data-scrollspy-anchor` must match a section `id`. `targetRef` is the scrollable container (defaults to the window). `smooth` defaults to true: pass `smooth={!reduceMotion}` from motion/react's `useReducedMotion()`. Mark the active link `aria-current="location"` (track it with `onUpdate`); `"page"` is for route links.
 
 ## frame
 
@@ -323,18 +327,16 @@ const [value, setValue] = useState<DateSelectorValue | undefined>()
 
 ```tsx
 <Frame>
-  <FramePanel>
   <FrameHeader>
     <FrameTitle>Title</FrameTitle>
     <FrameDescription>Description</FrameDescription>
   </FrameHeader>
-    <div className="p-5">Content</div>
+  <FramePanel>Content</FramePanel>
   <FrameFooter>Footer</FrameFooter>
-  </FramePanel>
 </Frame>
 ```
 
-**Gotcha:** a structured card shell for tool-like surfaces. `stacked` connects multiple panels with shared borders; `dense` removes panel padding; radius via the `--frame-radius` CSS variable.
+**Gotcha:** the shell for data and tool-like surfaces. `spacing` (`xs` | `sm` | `default` | `lg`) sets panel padding and the gap between panels, so pick it instead of padding classes (the 1 exception: `<FramePanel className="p-0">` around a `data-grid`); `variant` is `default` | `inverse` | `ghost`; `stacked` joins panels with shared borders; `dense` removes the frame's own inset and gap (panels sit flush), and panel padding stays; `--frame-radius` is the one radius knob (panels derive `--frame-panel-radius`).
 
 ## icon-stack
 
@@ -380,7 +382,7 @@ const [value, setValue] = useState<DateSelectorValue | undefined>()
 </CodeBlock>
 ```
 
-**Gotcha:** the one-liner is already a complete block; every child (header, title, language label, copy button, wrap toggle, expand button, line actions) is optional chrome, at any depth. Shiki is the only npm dependency and loads lazily, one chunk per language, on first highlight; `lines` (from `highlightCode` in `code-block-highlight`, which is server-safe and has no `"use client"`) or `highlight={false}` loads nothing at all. `maxLines` caps the height AND marks the block collapsible, which is what makes `CodeBlockExpandButton` appear. Use `variant="ghost"` when the block sits inside a surface that already has a border.
+**Gotcha:** the one-liner is already a complete block; every child is optional chrome, at any depth. Shiki is the only npm dependency and loads lazily on first highlight; `lines` (from `highlightCode` in `code-block-highlight`, which is server-safe and has no `"use client"`) or `highlight={false}` loads nothing at all. `maxLines` caps the height AND marks the block collapsible, which is what makes `CodeBlockExpandButton` appear. Use `variant="ghost"` when the block sits inside a surface that already has a border.
 
 ## alert
 
@@ -396,7 +398,7 @@ const [value, setValue] = useState<DateSelectorValue | undefined>()
 </Alert>
 ```
 
-**Gotcha:** shadcn-compatible API. `variant`: `default | destructive | info | success | warning | invert`. The non-default variants use ReUI extended color tokens (`--success`/`--info`/`--warning`/`--invert`), which the install adds. Defer generic alert rules to the shadcn skill.
+**Gotcha:** shadcn-compatible API. `variant`: `default | destructive | info | success | warning | invert`. The non-default variants use ReUI extended color tokens (`--success`/`--info`/`--warning`/`--invert`), which the install adds.
 
 ## badge
 
@@ -404,25 +406,116 @@ const [value, setValue] = useState<DateSelectorValue | undefined>()
 **Shape:**
 
 ```tsx
-<Badge variant="success-light" size="sm">Success</Badge>
+<Badge variant="success-light">Success</Badge>
-<Badge variant="outline" radius="full">Pill</Badge>
+<Badge variant="outline">Draft</Badge>
 ```
 
-**Gotcha:** shadcn-compatible. Rich `variant` set (solid, `-outline`, `-light` per color), `size` `xs..xl`, `radius` `default | full`. Like `alert`, the color variants rely on ReUI extended tokens. Prefer `Badge` variants over raw color classes for statuses.
+**Gotcha:** shadcn-compatible. Rich `variant` set (solid, `-outline`, `-light` per color), `size` `xs..xl` (keep the default), `radius` `default | full`. Like `alert`, the color variants rely on ReUI extended tokens. Prefer `Badge` variants over raw color classes: `<tone>-light` for a state, `outline` for a neutral label; an omitted `variant` is the solid primary fill.
+
+## color-picker
+
+**Required:** none (`<ColorPicker />` alone is complete); wire `value` + `onValueChange`.
+**Shape:**
+
+```tsx
+<ColorPicker value={color} onValueChange={setColor} alpha>
+  <ColorPickerTrigger />
+  <ColorPickerContent>
+    <ColorPickerPanel>
+      <ColorPickerArea />
+      <ColorPickerSlider channel="hue" />
+      <ColorPickerSlider channel="alpha" />
+      <ColorPickerFields />
+    </ColorPickerPanel>
+  </ColorPickerContent>
+</ColorPicker>
+```
+
+**Gotcha:** the value is a CSS color string in `format` (`"hex"` default, `"rgb"`, `"hsl"`, `"hsb"` or `"oklch"`), `null` when empty, never an object. `onValueChange` fires on every drag move, so save in `onValueCommit`. Without `alpha` the value is always opaque. A bare `ColorPickerPanel` inside the root is an inline picker; `ColorPickerInput` replaces the trigger with a typable field.
+
+## emoji-picker
+
+**Required:** none; wire `onEmojiSelect` to insert a pick, or `value` + `onValueChange` to use it as a field.
+**Shape:**
+
+```tsx
+<EmojiPicker onEmojiSelect={({ value }) => insert(value)}>
+  <EmojiPickerTrigger />
+  <EmojiPickerContent>
+    <EmojiPickerHeader>
+      <EmojiPickerSearch />
+    </EmojiPickerHeader>
+    <EmojiPickerCategories />
+    <EmojiPickerList />
+    <EmojiPickerFooter>
+      <EmojiPickerPreview />
+      <EmojiPickerSkinTone />
+    </EmojiPickerFooter>
+  </EmojiPickerContent>
+</EmojiPicker>
+```
+
+**Gotcha:** the emoji data is fetched at runtime from a pinned jsDelivr path (`EMOJI_PICKER_DATA_URL`): under a strict Content Security Policy allow it in `connect-src`, self-host with `dataUrl`, or pass `data` (from `normalizeEmojibaseData`). Store the pick's `value` (the native emoji in its skin tone, or `:shortcode:` for a custom one). `multiple` makes the value a list, capped by `max`.
+
+## file-type
+
+**Required:** none; pass `type` (an extension, a family name or any string).
+**Shape:**
+
+```tsx
+<FileType type="pdf" />
+<FileType type={getFileType(file.name)} size="sm" aria-hidden="true" />
+```
+
+**Gotcha:** one `span` with `role="img"`, no children, no `"use client"` (fine in a Server Component). Resolve a file name, path, URL or MIME type with `getFileType`, never by splitting on the dot. `variant`: `outline` (default) | `solid` | `light` | `mono`; `size`: `mini` | `xs` | `sm` | `default` | `lg` | `xl` (18 to 56px); recolor with a `text-*` class. Pass `aria-hidden="true"` when the file name sits beside it.
+
+## signature-pad
+
+**Required:** `SignaturePad` > `SignaturePadArea`
+**Shape:**
+
+```tsx
+<SignaturePad value={strokes} onValueChange={setStrokes}>
+  <SignaturePadArea>
+    <SignaturePadGuide />
+    <SignaturePadPlaceholder />
+    <SignaturePadControls>
+      <SignaturePadUndo />
+      <SignaturePadClear />
+    </SignaturePadControls>
+  </SignaturePadArea>
+</SignaturePad>
+```
+
+**Gotcha:** the value is an array of strokes (`{ points: [x, y, size][], color? }`), not an image: store it as JSON, and export with `useSignaturePad()` or `apiRef` (`toSVG`, `toDataURL`, `toBlob`) or `SignaturePadSave` (`format`, `onSave`). `onValueChange` fires per finished stroke, never per pointer move. Size the area with a class (`h-40` by default); an overlay in it that is not a control needs `pointer-events-none`. Drawing needs a pointer, so offer a typed name where a signature is required.
+
+## time-picker
+
+**Required:** none (`<TimePicker />` alone is complete); wire `value` + `onValueChange`.
+**Shape:**
+
+```tsx
+<TimePicker value={time} onValueChange={setTime}>
+  <TimePickerTrigger />
+  <TimePickerContent>
+    <TimePickerColumns />
+    <TimePickerFooter>
+      <TimePickerNow />
+      <TimePickerClear />
+      <TimePickerConfirm />
+    </TimePickerFooter>
+  </TimePickerContent>
+</TimePicker>
+```
+
+**Gotcha:** the value is a 24-hour `"HH:mm"` string (`"HH:mm:ss"` with `granularity="second"`) or `null`, never a `Date`, whatever `hourCycle={12}` shows. `minuteStep`, `min`, `max` and `isTimeDisabled` narrow the options; keep `isTimeDisabled` stable (module scope or memoized). `requireConfirm` holds picks as a draft until OK. `TimePickerInput` replaces `TimePickerTrigger` for a typable field, never beside it.
 
 ## base vs radix - write for the project's base
 
-ReUI ships every component in two builds: `base` (Base UI) and `radix` (Radix UI). The install command and name are identical, and the CLI installs the build matching the project. But you must write/adapt code against the **right base**, because their APIs differ.
+Every component ships a `base` (Base UI) and a `radix` (Radix UI) build under 1 name, and the CLI installs the one the project uses. Read `components.json` `style`: the part before the first `-` is the base (`base-nova` is **Base UI**, `radix-nova` is **Radix UI**). Write against that base; the deltas mirror shadcn's:
-
-**Detect the base first.** Read `components.json` -> `style` and take the segment before the first `-`:
-
-- `"style": "base-nova"` -> **Base UI**
-- `"style": "radix-nova"` -> **Radix UI**
-
-**Then use that base's API.** The deltas mirror shadcn's base-vs-radix split:
 
 - Slot/composition: Base UI `render={<… />}` vs Radix `asChild`.
 - `Select`: Base UI takes `items`; Radix uses `<SelectItem>` children.
 - `ToggleGroup`: Base UI `multiple` boolean vs Radix `type="single" | "multiple"`.
 
-The safest path is to **read the installed files and `c-*` examples** - they're already in your base, so reuse their wiring instead of guessing. When `get_component`'s inline `api` or an example shows the other base's shape, translate it to your base (or `validate_usage` to confirm). Defer the generic base/radix mechanics to the shadcn skill.
+Installed files and `c-*` examples are already in your base: reuse their wiring. When an `api` or example shows the other base's shape, translate it (or `validate_usage` to confirm).

.agents/skills/reui/rules/compose.md

@@ -0,0 +1,62 @@
+# Compose a screen
+
+Read this before you build any screen or section, then write the router's 7-line plan in your reply (Surface, Archetype, Container family, Density, Regions, States, Motion) and build exactly that plan. Style it with the topics the screen touches: [layout.md](./layout.md) (house defaults), [color.md](./color.md), [motion.md](./motion.md).
+
+If the brief names a layout or a motion (a left nav, 2 entrances), follow the brief and name the default you overrode. Several entrances run as 1 stagger: at most 6 items, 50ms apart, never on table rows or lists of 20 or more.
+
+## Surface
+
+- **Operational**: a screen people work in every day (dashboard, records, settings, admin tools). Frame-led, dense (`size="sm"` on controls that take it), no hero, no page-load motion.
+- **Marketing**: a page a visitor reads once (landing, pricing, features). Card-led sections on the spine below, default density.
+- **Flow**: a task people finish and leave (sign in, onboarding, checkout). 1 centered Card or a Dialog, 1 primary action per step.
+
+## Archetypes: pick 1 per screen
+
+- **Vertical rail settings** (3 to 6 sections): a section rail on the left, 1 content column, 1 section at a time; the rail turns horizontal below `md`.
+- **Horizontal tabs** (5 to 8 views of 1 object): a full-width `Tabs` rail under the header that scrolls sideways when it overflows, 1 content column.
+- **Frame stack**: stacked Frames, each 1 focused group of settings or records; a flat page split by `Separator` rows when it should feel lighter.
+- **Table-backed management**: header, toolbar (search, `filters`), then a `data-grid` with row actions and pagination, all in 1 Frame.
+- **Dashboard grid**: a KPI row, then 1 or 2 chart or list panels, then the table.
+- **Board**: header and filters, then a `kanban` board; `KanbanBoard` stacks its columns below `sm`, and a wider board scrolls inside its own region.
+- **Record detail**: header (title, status `Badge`, actions), a main column of sections, a side column of facts that stacks under it below `lg`.
+- **Marketing spine**: optional `Badge` eyebrow, headline (the only loud element), 1-clause subhead, action row (1 primary `Button`, a second `variant="outline"`, a third a text link), specific proof beside it ("27 reviews"), media. Omit parts; never reorder them.
+- **Auth card**: 1 centered `Card` at `max-w-sm`: title, fields, 1 full-width primary `Button`, a text link to the other flow.
+- **Wizard dialog**: a `Dialog` holding a `stepper`, 1 step visible, Back (`variant="outline"`) and Next (primary) in its footer.
+
+An app shell around any of them is the shadcn `Sidebar`, which turns into a `Sheet` below `md` by itself.
+
+## Fill each region in this order
+
+1. The ReUI component that owns the job: the router's Owners (`data-grid`, `filters`, `kanban`), `useFileUpload` from `@/hooks/use-file-upload` for file drop, or the one whose contract in [components.md](./components.md) matches.
+2. A shadcn/ui component (`Tabs`, `Dialog`, `AlertDialog`, `Sheet`, `Empty`, `Skeleton`, `sonner`).
+3. A free `c-*` example from `get_examples` that already composes it: install it and adapt.
+4. Your own composition of those parts. Never a hand-rolled table, modal or tabs.
+
+Import ReUI parts file by file: `data-grid`, `filters`, `cascader`, `event-calendar` and `gantt` are folders with no index file. `data-grid`: the import line in [components.md](./components.md). `filters`, each from `@/components/reui/filters/<file>`: `Filters` from `filters`, `createFilterQuery` and `flattenFilterConditions` (1 argument, the query) from `filters-query`, the types `FilterField` and `FilterQuery` from `filters-types`.
+
+## Wiring that works
+
+- `data-grid`: columns are `ColumnDef<DataGridFeatures, Order>[]`, a sortable header renders `DataGridColumnHeader`, and server paging needs `manualPagination: true` and `rowCount`.
+- `filters`: apply each condition by its `operator`. Negate swaps a rule to its inverse (`is` and `is_not`, `contains` and `not_contains`, `is_any_of` and `is_none_of`, `empty` and `not_empty`) and sets `negated` only when there is none: honour both.
+- A Base UI `Button` that navigates: `<Button nativeButton={false} render={<Link href="/x" />}>`.
+- shadcn's `CommandDialog` has no `Command` inside, unlike older shadcn: wrap `CommandInput` and `CommandList` in one. `CommandItem` search matches its `value`: omit it (text is used) or use the label; ids in `onSelect`.
+
+## Headers
+
+A page or panel header puts the title (1 to 3 words) and a 1-clause description on the left and the actions on the right: 1 primary `Button`, the rest `variant="outline"`, row actions `variant="ghost"`.
+
+## Responsive
+
+- No horizontal page scroll at 390px: grids drop to 1 column, side columns stack, `min-w-0` on every shrinking parent.
+- A KPI row is `grid-cols-1 sm:grid-cols-2 xl:grid-cols-4`.
+- Every header and toolbar action stays reachable at every width: a control hidden below a breakpoint moves into a menu shown only there.
+- A table scrolls sideways inside its Frame, never the page (the `data-grid` contract in [components.md](./components.md)).
+
+## States and motion
+
+- Each data region owns its states: empty (shadcn `Empty` with a title, 1 line and the primary action), loading (`Skeleton` in the real layout; `data-grid` takes `isLoading`) and error (inline in the region, with Retry). Never 1 spinner for the whole page.
+- Motion: none, or the 1 entrance per view; installed parts keep their own motion, and hover and press follow [motion.md](./motion.md). An Alert, field error, notice or data that appears after an action or a load is a reveal, not an entrance: fade it in with the reveal row.
+
+## Remove first
+
+Overloaded? Remove, in order: decorative summaries, duplicate context, extra badges, long descriptions, secondary actions, mismatched icon treatments. Add a region only when the brief needs it.

.agents/skills/reui/rules/icons.md

@@ -1,18 +1,18 @@
 # Icons (ReUI delta over shadcn)
 
-Follow the shadcn icon rules (use the project's configured `iconLibrary`, `data-icon` on icons inside `Button`, no sizing classes on icons inside components, pass icons as component objects not string keys). ReUI adds the following.
+Use the project's configured `iconLibrary`, put `data-icon="inline-start"` on an icon before the label inside `Button` (`data-icon="inline-end"` on one after it), put no sizing classes on icons inside components, and pass icons as components, not string keys. ReUI adds the following.
 
 ## Portable icons (library-agnostic)
 
-ReUI components, examples, and blocks are authored to be icon-library-agnostic. When `iconLibrary` is set in `components.json`, the shadcn CLI installs each item's icons in **your** library automatically - you swap nothing. If an installed item's icons don't match your project (for example `iconLibrary` isn't set, so they came in from the item's demo library), change the **import source and component name** to your library, keeping the same icon-name semantics:
+ReUI components, examples, and blocks are authored to be icon-library-agnostic. The shadcn CLI installs each item's icons in the library named by `iconLibrary` in `components.json` - you swap nothing. When `iconLibrary` is not set, the CLI falls back to `lucide`, so the icons arrive as `lucide-react` imports. If that is not the project's library, set `iconLibrary` before installing, or change the **import source and component name** to your library, keeping the same icon-name semantics (each key below is an `iconLibrary` value):
 
 - `lucide` -> `lucide-react`
 - `tabler` -> `@tabler/icons-react`
 - `phosphor` -> `@phosphor-icons/react`
-- `remix` -> `@remixicon/react`
+- `remixicon` -> `@remixicon/react`
-- `hugeicons` -> `@hugeicons/react`
+- `hugeicons` -> icons from `@hugeicons/core-free-icons`, rendered through `HugeiconsIcon` from `@hugeicons/react`
 
-Don't assume `lucide-react`; read `iconLibrary` from `components.json`.
+Don't assume `lucide-react`: read `iconLibrary` from `components.json`, and when it is unset, check which icon package the project already uses.
 
 ## Keep icons purposeful
 
@@ -20,20 +20,24 @@ Icons support the hierarchy, they don't replace it: keep them small, matched to
 
 ## Motion Icons (the `@reui/icons/...` set)
 
-ReUI ships its own icon set in 4 styles (outline, solid, duotone, filled), each icon in two variants:
+ReUI ships its own icon set in 4 styles (outline, solid, duotone, filled). Every icon installs from the default path; some also have a real hover animation (motion/react) on the animated path:
 
 ```bash
-npx shadcn@latest add @reui/icons/default/<style>/<name> --yes    # static
+npx shadcn@latest add @reui/icons/default/<style>/<name> --yes    # static, every icon
-npx shadcn@latest add @reui/icons/animated/<style>/<name> --yes   # hover-animated (motion/react)
+npx shadcn@latest add @reui/icons/animated/<style>/<name> --yes   # hover-animated, only when the icon has installAnimated
 ```
 
-Motion Icons need an Ultimate license for **both discovery and install** (`REUI_LICENSE_KEY`, see [cli.md](./cli.md)). Without it `search_icons` comes back `locked` with no results and `search` returns no icons at all, so check the plan before you promise the user icons: on a free or Pro account, use the project's own `iconLibrary` instead. Reach for a Motion Icon on a primary action when a subtle hover cue helps; keep motion restrained.
+The animated path answers for every icon, but for an icon without a real animation it installs the same static drawing, so only use it when the icon carries `installAnimated`.
+
+Motion Icons need an Ultimate license for **both discovery and install** (`REUI_LICENSE_KEY`, see [cli.md](./cli.md)). Without it `search_icons` comes back `locked` with no results and `search` returns no icons at all, so check the plan before you promise the user icons: on a free or Pro account, use the project's own `iconLibrary` instead. On Ultimate, a Motion Icon on the primary action is the 1 hover flourish (see [motion.md](./motion.md)).
 
 Finding icons:
 
-- Several icons (the common case): **`search_icons(concepts[])`** - up to 24 concepts in one call, the best icons per concept with install commands. Pass `animated: true` to get only icons with a hover-animated Motion variant.
+- Several icons (the common case): **`search_icons(concepts[])`** - up to 24 concepts in one call, the best icons per concept with install commands. Pass `animated: true` to get only icons with a real hover animation; a concept with none comes back empty and `animatedNote` says so.
 - One icon: `search` with `type: "icon"`.
-- Icon results and `get_icon` carry `animated: true` and `installAnimated` when an animated variant exists - use those install strings, do not construct paths by hand.
+- Icon results and `get_icon` carry `animated: true` and `installAnimated` only when the icon really animates - use those install strings, do not construct paths by hand.
-- Every icon result carries a `previewUrl` (its live icon-category page) - **share it with the user** so they can SEE the icon before installing.
+- Every icon result carries a `previewUrl` (the icon's own page) and a `thumbnail` (its 4 styles side by side) - **show or share them** so the user can SEE the icon before installing, and look at the thumbnail yourself before trusting a name.
+- Several icons can fit one concept. Before installing, read `get_icon`'s `whenNotToUse` (each entry names the icon to use instead), and use its `label` as the `aria-label` of an icon-only button.
+- Icon names were revised in the icons v2 update. An old name (`save-2`, `abstract-18`) still answers in `get_icon`, `get_install_command`, `validate_usage` and `get_thumbnail` with the icon it became and `renamedFrom`, and still installs that same drawing; write the current name in new code. That includes names renamed because they misdescribed the drawing (`watch` installs a sand timer, `chevrons-down` points up), so look at the `thumbnail` before trusting an old name. `search_icons` answers an old name the same way, unless it is also an ordinary word icons are named for today (`watch`): then it ranks today's icons for the word, and `oldNameNotes` names the icon the old name installs. `search` ranks by current names only, so look an old name up with `get_icon`, never `search`. Only `tag` (it drew a money bag) no longer answers: the reply names the icons to use instead.
 
 The `icon-stack` component composes multiple icons into a stacked display.

.agents/skills/reui/rules/layout.md

@@ -0,0 +1,56 @@
+# Layout: house defaults, hierarchy, spacing, surfaces and states
+
+## Point of view
+
+Pick an emotional register before you compose - calm, operational, premium, editorial, dense, energetic - and let layout, spacing, surface treatment, and icon behavior all reinforce it. One or two memorable decisions and restraint everywhere else beats ten generic ones. UI with no point of view reads as generated.
+
+**The bar:** before you finish, ask: would a product team keep this instead of replacing it? Does it still feel strong after swapping in real content? If not, reuse the shipped ReUI design harder - don't restyle it into something new - then run the [quality.md](./quality.md) gates.
+
+## House defaults
+
+<!-- gen:house-defaults:start -->
+
+- **Spacing ladder**: `gap-1` to `gap-2` in a row, `gap-3` to `gap-4` between groups, `gap-6` to `gap-8` between sections, `gap-0.5` title to description. Deviate when: skip a level if you like, never invert them.
+- **Panel title**: no size class on `FrameTitle` or `CardTitle`. Deviate when: never; `FrameTitle` ships `text-sm font-semibold`, `CardTitle` follows the installed style.
+- **Meta text**: `text-xs`. Deviate when: never smaller, except inside illustrative mini-UI artwork.
+- **Frame spacing**: `spacing="default"`, written out. Deviate when: `sm` in a sheet, dialog, drawer or popover, or for a dense data grid in a panel; `xs` or `lg` only with a stated reason.
+- **Leading row icon**: `size-4`. Deviate when: never in a glyph column; `Spinner` ships `size-4`, so a loading swap keeps the width.
+- **Icon inside a control**: no size class; the control sizes it per style and size. Deviate when: never pin `size-*` on an svg inside `Button`, `InputGroupButton`, `InputGroupAddon`, `DropdownMenuItem`, `DropdownMenuCheckboxItem`, `DropdownMenuRadioItem`, `DropdownMenuSubTrigger`, `SelectItem`, `CommandItem`, `Toggle`, `ToggleGroupItem`, `IconTile` or `MarkerIcon`.
+- **Badge size**: `default` (leave `size` off). Deviate when: `lg` or `xl` only when the user asks; never `sm` or `xs`.
+- **Neutral badge**: `variant="outline"`. Deviate when: never `secondary`, `default` or `primary-light` for an id, tag, version, count or Draft/Beta label; inline after the title, never `ml-auto`.
+- **State badge**: `<tone>-light`: `success-light`, `info-light`, `warning-light`, `destructive-light`; `primary-light` for brand or pending. Deviate when: never the solid tone, `secondary` or `outline` for a state; 1 state badge per row.
+- **Solid badge**: never unless asked; always name a variant (an omitted one is the solid primary fill). Deviate when: only when the user asks for a solid fill (`default`, `secondary`, `success`, `info`, `warning`, `destructive`, `focus`, `invert`).
+- **Slot padding**: the primitive's own inset; set density with `Frame spacing=` or `Card size="sm"`. Deviate when: never a padding class on `CardHeader`, `CardContent`, `CardFooter`, `FrameHeader`, `FramePanel`, `FrameFooter`, `SheetHeader`, `SheetContent`, `SheetFooter`, `DialogHeader`, `DialogContent`, `DialogFooter`, `PopoverContent`, `DropdownMenuContent`, `TabsContent` or `AccordionContent`.
+- **Inline separator**: `<span aria-hidden className="bg-muted-foreground/40 size-1 shrink-0 rounded-full" />`. Deviate when: never a typed `·` between rendered segments; a literal dot only inside 1 data string.
+- **List rows**: dense, auto height; a row with no second line stays 1 tight line. Deviate when: never uniform height.
+- **Current list item**: ghost hover fill (`bg-muted`, `dark:bg-muted/50`) plus `aria-current`. Deviate when: never a dot or a check mark.
+
+<!-- gen:house-defaults:end -->
+
+## Hierarchy
+
+- 1 focal point per card or panel: the dominant metric or task, then its label, then detail. Borders, separators and surfaces make 2 to 3 bands, never equal weight everywhere.
+- Overloaded? Remove, in order: decorative summaries, duplicate context, extra badges, long descriptions, secondary actions, mismatched icon treatments.
+
+## Spacing and copy
+
+- Siblings take the parent's `gap-*`, never their own margins; a heading sits nearer its own content than the section above it. Never pad an operational surface like a marketing page.
+- Titles 1 to 3 words, a description 1 clause (about 8 words), a context line 2 segments at most, labels and buttons 1 to 2 words, a toast title 4 words at most. Copy that wraps at desktop width is over budget: cut it, never shrink the type.
+
+## Surfaces
+
+- `Frame` for data and operational surfaces (tables, lists, tool panels); `Card` for bounded sections and marketing.
+- Prefer 1 container family per screen. With a KPI row and a table, use FramePanels for both, or Cards for the KPIs and a Frame for the table; never a Card inside a Frame or a Frame inside a Card.
+- A `data-grid` inside a Frame is the one place for `<FramePanel className="p-0">`; elsewhere no padding on slots. This is the 1 exception to the Slot padding row above.
+- 1 archetype per screen (operational ones and the marketing spine): [compose.md](./compose.md) lists each with its regions.
+
+## Type and buttons
+
+- A KPI value is `text-2xl font-semibold tracking-tight tabular-nums`.
+- `size="sm"` on dense surfaces, `variant="outline"` for secondary actions, `ghost` for row actions; 1 primary per region, 1 variant per toolbar.
+- A toolbar separator is `<Separator orientation="vertical" className="my-auto h-4" />`; at a 390px viewport a toolbar stays on 1 row.
+
+## States and responsive
+
+- Empty: a short message plus the primary action. Loading: a `Skeleton` in the real layout, never a centered spinner. Error: inline with a retry.
+- No horizontal page scroll at 390px: reflow to 1 column instead of shrinking; `min-w-0` on the shrinking parent and `truncate` on single-line labels.

.agents/skills/reui/rules/motion.md

@@ -0,0 +1,80 @@
+# Motion
+
+## Installed motion: never retime it
+
+<!-- gen:installed-motion:start -->
+
+| Installed part     | Duration          |
+| ------------------ | ----------------- |
+| Overlays           | 100ms             |
+| Tooltip            | 150ms (default)   |
+| Sheet, sidebar     | 200ms             |
+| Navigation menu    | 300ms, exit 150ms |
+| rhea control focus | 200ms             |
+
+<!-- gen:installed-motion:end -->
+
+Never add or change `duration-*`, `ease-*`, `animate-*` or `transition-*` on installed parts (`components/ui`, `components/reui`); Button already presses with `active:not-aria-[haspopup]:translate-y-px`.
+
+## House motion: your own code
+
+<!-- gen:house-motion:start -->
+
+| Use                       | Tailwind                                                |
+| ------------------------- | ------------------------------------------------------- |
+| Hover, press, color       | `transition-colors` (150ms, no duration or ease class)  |
+| Reveal or expand          | `duration-200 ease-[cubic-bezier(0.22,1,0.36,1)]`       |
+| Large surface (ceiling)   | `duration-300` with the same curve                      |
+| An overlay you build      | `duration-100` with the same curve                      |
+| Media hover zoom          | `duration-500` with the same curve, `scale-105` at most |
+| Width tied to the sidebar | `duration-200 ease-linear`                              |
+
+<!-- gen:house-motion:end -->
+
+- Press on a custom surface (not Button): `transition-transform active:scale-[0.98] motion-reduce:transition-none`.
+- Hover color plus press on 1 element: `transition-[color,background-color,border-color,scale]` (`transition-colors` and `transition-transform` override each other; `scale-*` sets `scale`).
+- motion/react takes the same values in seconds: `{ duration: 0.2, ease: [0.22, 1, 0.36, 1] }`; hover `{ duration: 0.15 }`.
+- Drag, reorder and layout use `const SPRING = { type: "spring", visualDuration: 0.25, bounce: 0 }`. Define `SPRING` only when something drags, reorders or toggles layout.
+
+## Hard rules
+
+- Animate `transform` and `opacity`; color only for hover and press. Never `transition-all`; width, height, top or left only in the sidebar row.
+- Never `animate-bounce` or a spring with bounce above 0. `ease-linear` only for progress, spinners, loops and the sidebar; never Tailwind's `ease-in`, `ease-out` or `ease-in-out`.
+- `motion/react` only: never framer-motion, GSAP or react-spring.
+- At most 1 entrance animation per view (a stagger counts as 1), and every animation has a reduced-motion path. Feedback that appears after an action or a load (an Alert, a field error, a saved notice, data replacing its `Skeleton`) is a reveal, not an entrance: fade it in with the reveal row.
+- Never add named `duration-*` utilities or redefine `--ease-out`.
+- Loading: a `Skeleton` (it already pulses) in the real layout; a spinner only inside a busy button.
+- Motion Icons (Ultimate only): 1 on the primary action is the 1 hover flourish; install it with its `installAnimated` command; never retime it.
+
+## Performance
+
+Each line names its lint rule, or judgement (no check).
+
+- Never animate blur, drop-shadow, backdrop-filter, box-shadow, border-radius, background-position, background-size or a custom property that feeds transform; clip-path and stroke-dashoffset only as a one-shot reveal of 300ms or less, or a loop on an element of 240x240px or less. Name the properties: never a bare `transition` (it also animates box-shadow and filter). (`reui/no-paint-animation`; 240px: judgement (no check))
+- A hover lift is `transform` over a static shadow, or a shadow layer whose `opacity` fades; never a transitioned `hover:shadow-*`. (`reui/no-paint-animation`)
+- CSS first: hover, press, reveals and ambient loops are Tailwind transitions, `tw-animate-css` or `@keyframes`. motion/react only for exit on unmount, drag, springs that follow input, `layout` and scroll-linked values. (judgement (no check))
+- In motion/react, `x`, `y`, `scale`, `rotate`, `pathLength` and every SVG target run on the main thread: never `repeat: Infinity` on them (loop with CSS `@keyframes`), and keep 20 or fewer JS-driven animations running per view. An opacity loop leaves the main thread only on an HTML element without `repeatDelay`, `repeatType: "mirror"` or `onUpdate`. (`reui/no-main-thread-loop`)
+- SVG geometry (a rect's `width`, a circle's `r`) does not reflow the page but runs in motion's JS loop: one-shot only. (`reui/no-main-thread-loop`)
+- Never set React state per frame (requestAnimationFrame, `useAnimationFrame`, `onUpdate`, scroll or pointermove listeners): bind styles to `useMotionValue`, `useTransform` or a CSS variable on the element. (`reui/no-frame-setstate`)
+- A JS-driven loop (motion `repeat`, requestAnimationFrame, canvas) pauses offscreen with `useInView` or IntersectionObserver; `content-visibility: auto` never stops script, and a setInterval loop also pauses on `visibilitychange`. (`reui/loop-offscreen`)
+- No permanent `will-change`: no `will-change-*` class or `willChange` style, except during a drag, removed when it ends. (`reui/will-change`)
+- Stagger 50ms or less per step (`staggerChildren: 0.05`), only the first 6 items, the whole entrance done in 500ms, first mount only, never on table rows or lists of 20 or more; scroll reveals use `whileInView` with `viewport={{ once: true }}`. (`reui/stagger-budget`; item count and tail: judgement (no check))
+- A height reveal (`grid-template-rows`) starts only on a click or key press, runs 200ms or less, 1 at a time; never `initial={{ height: 0 }}`. (`reui/no-layout-transition`)
+- The motion `layout` prop on 1 list at a time (about 20 elements), never in data-grid or virtualized rows; drag and drop goes through `@reui/sortable` or `@reui/kanban`. (judgement (no check))
+- AnimatePresence `mode="wait"` only with an exit of 150ms or less; `initial={false}` around content present at first render. (`reui/stagger-budget`)
+- Never hide the hero or the largest heading behind a motion/react `initial` at opacity 0 (hidden until JavaScript runs). (judgement (no check))
+- At most 3 ambient loops per view: CSS `@keyframes` on transform or opacity, a period of 2s or more, exempt from the duration table; once 3 run in a view they pause offscreen (a `content-visibility: auto` wrapper with `contain-intrinsic-size: auto <h>` where nothing paints outside it, or IntersectionObserver toggling `animation-play-state`). At most 50 animating elements per view (Skeletons, loaders and CSS opacity loops not counted). A static blur above 12px, at most 1 per view, only on a surface that neither moves nor sits over motion. (judgement (no check))
+
+## Reduced motion
+
+- Reduced motion is a substitute, not an off switch: replace movement (translate, scale, rotate, slide, zoom, layout, parallax, auto-scroll) with an opacity fade of 150ms or less or an instant change. Keep color and opacity feedback, focus, progress bars, loaders, the Skeleton pulse, chevrons and the switch thumb. (`reui/reduced-motion`)
+- The reduced state is the resting state: content never waits at opacity 0, hidden or offset for an animation that will not play; a count-up shows its final value. (`reui/reduced-motion`)
+- In every mode an entrance moves 8px or less and starts at scale 0.95 or more; nothing moves a surface larger than half the viewport. (`reui/reduced-motion`; half the viewport: judgement (no check))
+- `<MotionConfig reducedMotion="user">` once at the app root (its default ignores the OS) AND a `motion-reduce:` path on every CSS animation (MotionConfig never reaches CSS). An example or block wraps its own motion subtree. (`reui/reduced-motion`)
+- MotionConfig makes only transform keys and width, height, top and left instant; opacity, filter, clipPath, pathLength, color and a `transform` string keep animating, so a loop or those keys branch on `useReducedMotion()` and rest on the final value. (`reui/reduced-motion`)
+- `useReducedMotion()` is null on the server: never let it pick markup (JSX, className, `initial`) or reach `animate` under `initial={false}`; branch only animate, transition, `while*` or exit values. (`reui/reduced-motion-ssr`)
+- No SMIL (`<animate>`, `<animateMotion>`, `<animateTransform>`); canvas and requestAnimationFrame loops draw 1 resting frame under `prefers-reduced-motion: reduce`. (`reui/no-smil`, `reui/reduced-motion`)
+- Smooth scroll only under `motion-safe:scroll-smooth`. A focus ring appears at once: never `transition-all`, `transition-shadow` or a bare `transition` on it. (`reui/reduced-motion`, `reui/focus-transition`)
+- Loaders and typing dots keep their opacity pulse; every decorative loop gets `motion-reduce:animate-none`, never an inline `style={{ animation }}` (it beats the class). (`reui/reduced-motion`)
+- Anything that moves on its own for more than 5 seconds (autoplay, marquee, ticker) gets a visible Pause control, pauses on hover and focus, and never autoplays under reduced motion. (`reui/pausable-autoplay`)
+- Installed overlays keep their 100ms fade, zoom and 8px slide under reduced motion: disclosed, not edited; an app that needs WCAG 2.3.3 adds its own override. (judgement (no check))

.agents/skills/reui/rules/quality.md

@@ -1,6 +1,6 @@
-# Quality gates (security, accessibility, scroll)
+# Quality gates (security, accessibility, scroll, self-checks)
 
-These are the **done gate**, not a nice-to-have: before you call any ReUI work finished, call the MCP `get_audit_checklist` tool and pass every item below (plus the craft bar in [craft.md](./craft.md)). Then typecheck and lint.
+This is the **done gate**: before you call any ReUI work finished, call `get_audit_checklist` and pass every item below (plus the bar and the states in [layout.md](./layout.md)). Then run the self-checks, typecheck and lint, compare with [anti-patterns.md](./anti-patterns.md), and hand back in the 5 lines below.
 
 ## Security
 
@@ -10,13 +10,48 @@ These are the **done gate**, not a nice-to-have: before you call any ReUI work f
 
 ## Accessibility
 
-- Implicit list/card items that navigate get real anchors with a standard hover affordance.
+- Implicit list/card items that navigate get real anchors with a standard hover affordance. Hover on a display-only card: make the card a link to its detail, or give it no hover.
+- `aria-current="page"` on the current route link, `"location"` on the current in-page section link.
+- Destructive confirms use `AlertDialog`, not `Dialog`, and cannot be dismissed while pending.
 - Icon-only or numeric buttons need an `aria-label`; decorative icons get `aria-hidden`.
 - Every non-submit button is `type="button"`.
-- Keyboard + focus: everything interactive is reachable in a sensible Tab order with a visible focus ring; layers (dialogs/sheets/menus) trap focus and close on `Escape`. ReUI components ship standard keyboard behavior - read each component's inline `api` rather than re-implementing it.
+- Keyboard + focus: everything interactive is reachable in a sensible Tab order with a visible focus ring; layers (dialogs/sheets/menus) trap focus and close on `Escape`. ReUI components ship their keyboard behavior; never re-implement it.
-- Announce async UI: loading and error messages use `role="status"` / `aria-live` so they're not silent to screen readers.
+- Announce async UI: loading and error messages use `role="status"` or `aria-live`.
 
 ## Scroll mechanics
 
 - Make scroll regions with a parent-owned height: a `min-h-0` + flex chain down to the scroll container. Never guess a `max-h`.
 - The scroll container owns `overflow-auto`; ancestors stay `min-h-0` so the height resolves.
+
+## Self-checks
+
+After every UI edit, run these with ripgrep on `<files>`: each file you wrote or changed, by path, never `components/ui` or `components/reui`. A clean check prints nothing; a hit prints `file:line:code`. The last filter drops comment lines.
+
+```sh
+# C1 banned motion: transition-all, bare or paint transitions, layout keys, bounce, overshoot, scale 0, hover and tap gestures, will-change
+rg -nH 'transition-all|transition(-property)?: *[^;]*\b(all|width|height|top|left|margin|padding|box-shadow|filter)\b|[\x27\x60" ]transition( +(duration|ease|delay|hover|focus|active|group|motion)|[\x27\x60"])|transition-shadow|transition-\[[^\]]*\b(width|height|top|left|right|bottom|inset|margin|padding|filter|radius|background-position|background-size|all)\b|animate-bounce|will-change|willChange|bounce: *(0?\.0*[1-9]|[1-9])|[\x27"](backIn|backOut|backInOut|anticipate)[\x27"]|(cubic-bezier\(|ease: *\[)([^,]*,|([^,]*,){3})[ _]*(-|1\.[0-9]*[1-9]|[2-9])|scale: *0 *[,}]|\bscale-0\b|while(Hover|Tap|Focus)=|(initial|animate|exit)=\{\{[^}]*\b(width|height|maxHeight|top|left|margin|padding|boxShadow|filter|backdropFilter|borderRadius)[A-Za-z]*:' <files> | rg -v ':[0-9]+:\s*(//|/\*|\*(\s|/|$)|\{/\*)'
+# C2 off-scale motion: a duration off 0, 100, 150, 200 and 300 (duration-500 only on a media hover zoom), ease-in or ease-out, a stagger step above 0.05
+rg -nH 'duration-([1-9]|[1-9][0-9]|1(0[1-9]|[1-46-9][0-9]|5[1-9])|[23](0[1-9]|[1-9][0-9])|[4-9][0-9]{2}|[1-9][0-9]{3,})\b|duration-\[|\b([1-9]|[1-9][0-9]|1(0[1-9]|[1-46-9][0-9]|5[1-9])|[23](0[1-9]|[1-9][0-9])|[4-9][0-9]{2}|[1-9][0-9]{3,})ms\b|(^|[^0-9.])(0?\.(0[0-9]*|1[0-46-9][0-9]*|15[0-9]+|[23][0-9]+|[4-9][0-9]*)|1\.[0-9]+)s\b|duration: *([1-9]\b|0?\.(0[0-9]*|1[0-46-9][0-9]*|15[0-9]+|[23][0-9]+|[4-9][0-9]*)\b)|\bease-(in|out|in-out)\b|(staggerChildren: *|stagger\( *)(0?\.0[6-9]|0?\.[1-9]|[1-9])|delay: *[^,}]*\b(i|index|idx)\b *\* *(0?\.0[6-9]|0?\.[1-9]|[1-9])|delay: *(0?\.0[6-9]|0?\.[1-9]|[1-9])[0-9.]* *\* *\b(i|index|idx)\b' <files> | rg -v ':[0-9]+:\s*(//|/\*|\*(\s|/|$)|\{/\*)'
+# C2 read by hand: a duration or delay that is not a number, every index delay (allowed: 0.05 or less on the first 6 items, as Math.min(i, 5) * 0.05, never on table rows)
+rg -nH '([dD]uration|[dD]elay): *[^0-9 .]|delay: *[0-9.]+ *\*' <files> | rg -v ':[0-9]+:\s*(//|/\*|\*(\s|/|$)|\{/\*)'
+# C3 state color: raw palette on a line that names a state
+rg -nH '\b(text|bg|border|ring|fill|stroke)-(red|rose|orange|amber|yellow|lime|green|emerald|teal|cyan|sky|blue|indigo|violet)-[0-9]{2,3}\b' <files> | rg -i ':[0-9]+:.*\b(success|paid|active|completed?|done|approved|online|healthy|live|error|failed|critical|overdue|declined|rejected|cancell?ed|blocked|expired|warning|pending|degraded|due|info|processing|running|scheduled|queued)\b' | rg -v ':[0-9]+:\s*(//|/\*|\*(\s|/|$)|\{/\*)'
+# C4 structure: a raw table, select or fixed overlay, framer-motion, Math.random() or Date.now()
+rg -nH '<table\b|<select\b|fixed inset-0|from ["\x27]framer-motion|Math\.random\(|Date\.now\(' <files> | rg -v ':[0-9]+:\s*(//|/\*|\*(\s|/|$)|\{/\*)'
+# C4 on a base-* style only (skip on radix-*): Radix asChild
+rg -nH '\basChild\b' <files> | rg -v ':[0-9]+:\s*(//|/\*|\*(\s|/|$)|\{/\*)'
+```
+
+Fix every hit, run again, stop after the second run and list what is left with file and line. A C4 hit stays only when a comment in the file says why. These also stay, named on hand-back line 2: C1 on the motion.md sidebar row or on will-change set in a drag handler and removed when it ends; C2 on a media hover zoom; `Math.min(i, 5) * 0.05`.
+
+No `rg` (a missing one also prints nothing): run each first pattern with your own search tool, skipping comment lines. No shell: read each changed file for what each `#` line names.
+
+## Hand-back
+
+End your reply with these 5 lines:
+
+1. Installed: each item with its `previewUrl`.
+2. Motion: each animated element and its [motion.md](./motion.md) table row, or "none".
+3. States: covered, or which are missing.
+4. Checks: C1 to C4, screenshots or "not visually verified", the audit checklist, typecheck, lint; each one run, or "not run" and why.
+5. Deviations: any rule broken or check hit left, with file:line and why.

.agents/skills/reui/rules/registry.md

@@ -2,14 +2,14 @@
 
 ReUI is a shadcn-compatible registry with four entity types. **Examples and blocks are built FROM components** - reuse them, don't rebuild.
 
-- **component** - one of the 22 ReUI building blocks with a real API (`data-grid`, `kanban`, `filters`, `date-selector`, `tree`, ...). Install directly (`@reui/data-grid`) or let it come in as a dependency of an example/block. Free. Read its API with `get_component(name)`.
+- **component** - one of the ReUI components with a real API (`data-grid`, `kanban`, `filters`, `date-selector`, `tree`, ...). Install directly (`@reui/data-grid`) or let it come in as a dependency of an example/block. Free. Read its API with `get_component(name)`.
 - **example** - a free `c-*` single-pattern use-case of a component (`c-kanban-1`, `c-data-grid-3`). Install one and read it to copy real composition. Find a component's examples with `get_examples(name)`.
-- **block** - a premium, full-page section that composes several components (`data-grid-base-2`, `settings-2`). Pro or Ultimate license at install. Adapts to your active theme via semantic tokens.
+- **block** - a premium, full-page section that composes several components (a full dashboard or settings page). Pro or Ultimate license at install. Its surfaces, text and borders use your theme tokens; its category accents (chart series, tags, avatars) are fixed palette colors paired for light and dark.
-- **icon** - Motion Icons in 4 styles (outline, solid, duotone, filled), static (`@reui/icons/default/<style>/<name>`) and hover-animated (`@reui/icons/animated/<style>/<name>`). Ultimate license at install. See [icons.md](./icons.md).
+- **icon** - Motion Icons in 4 styles (outline, solid, duotone, filled), static (`@reui/icons/default/<style>/<name>`), plus a hover-animated variant (`@reui/icons/animated/<style>/<name>`) only for icons that carry `installAnimated`. Ultimate license at install. See [icons.md](./icons.md).
 
 ## The @reui registry
 
-Install everything through the shadcn CLI: `npx shadcn@latest add @reui/<name> --yes`. The CLI reads the `@reui` registry from the project's `components.json`. Free items need only the plain string form:
+ReUI items install through the shadcn CLI as `npx shadcn@latest add @reui/<name> --yes`, with `<name>` exactly as a ReUI tool returned it (`search`, a getter, `get_install_command`). The CLI reads the `@reui` registry from the project's `components.json`. Free items need only the plain string form:
 
 ```json
 { "registries": { "@reui": "https://reui.io/r/{style}/{name}.json" } }
@@ -17,18 +17,27 @@ Install everything through the shadcn CLI: `npx shadcn@latest add @reui/<name> -
 
 Premium installs need the authenticated form + `REUI_LICENSE_KEY` in `.env.local` - see [cli.md](./cli.md). The MCP `get_project_context` tool returns the right config.
 
+## ReUI items vs shadcn/ui components
+
+- **ReUI items** (the components, `c-*` examples, blocks, Motion Icons) install as `@reui/<name>`.
+- **shadcn/ui components** that ReUI builds on (`button`, `dialog`, `select`, `card`, ...) are not ReUI items. They install from shadcn by bare name: `npx shadcn@latest add button --yes`.
+- **`badge` and `alert` exist in both registries.** `@reui/badge` and `@reui/alert` are ReUI's components (the extra variants in [components.md](./components.md)); bare `badge` and `alert` are shadcn's.
+- **In an item's `registryDependencies`**, a `@reui/<name>` entry is a ReUI item and a bare name is a shadcn/ui item. `add` installs both for you.
+- **Templates are not registry items.** Download one from its page on reui.io (https://reui.io/templates?ref=skill); no `shadcn add` command installs a template.
+
 ## Know your base: base or radix
 
 ReUI ships every item in two builds - `base` (Base UI) and `radix` (Radix UI) - with mirrored names. The CLI installs the build matching your project automatically, but **you must write code against the right base's API**. Detect it from `components.json` -> `style`: the segment before the first `-` is the base (`base-nova` -> Base UI, `radix-nova` -> Radix UI). The installed files and `c-*` examples are already in your base - read them and adapt on that base. See [components.md](./components.md) for the API deltas.
 
 **So the MCP's own `docsUrl` and `previewUrl` match your base**, send your `style` to the MCP: add `?style=<your components.json style>` to the ReUI MCP server URL (or set an `X-Reui-Style` header) in your MCP client config - set once, applies to every call. The MCP then resolves docs/preview links to YOUR library (`/docs/components/radix/...`, `/preview/radix/...` for a radix project) instead of the default base; `get_project_context` echoes back the style it currently sees so you can confirm it. Install commands are unaffected (the CLI already installs the right variant). If you notice the MCP returning `base` links for a `radix` project, tell the user to add `?style=` to the server URL.
 
-Blocks adapt to your active theme through semantic tokens and CSS variables - change the theme and every block follows.
+Blocks take surfaces, text and borders from your theme tokens, so a theme change re-tints them; category accents stay fixed by design.
 
 ## Free vs premium
 
-- **Free, no license key:** the 22 components, all `c-*` examples, and this skill. The MCP is free too but still needs a free ReUI account (the agent signs in on first use); a license key is only for premium installs.
+- **Free, no license key:** the ReUI components, all `c-*` examples, and this skill. The MCP is free too but still needs a free ReUI account (the agent signs in on first use); a license key is only for premium installs.
-- **Premium, license required at install:** blocks (Pro or Ultimate), Motion Icons and templates (Ultimate). Set `REUI_LICENSE_KEY` (see [cli.md](./cli.md)).
+- **Premium, license required at install:** blocks (Pro or Ultimate) and Motion Icons (Ultimate). Set `REUI_LICENSE_KEY` (see [cli.md](./cli.md)).
+- **Templates** are downloads, not installs: free ones need only a ReUI account, and each template's page on reui.io says what the others need.
 
 ## Component API index
 

.agents/skills/reui/rules/theming.md

@@ -0,0 +1,37 @@
+# Theming: styles, radius, density and sizes
+
+## Read your style
+
+`components.json` `style` is `<base>-<style>`: the part after the first `-` names the style (`base-nova` is nova on Base UI). The style is baked into `components/ui` and `components/reui` at install, so the installed files hold your real values: read your installed `button.tsx` (usually `components/ui/button.tsx`) for heights, radius and padding before you build anything that sits beside a control.
+
+## The style ladder
+
+<!-- gen:style-ladder:start -->
+
+| Style | Button `sm` / default / `lg` | Button radius  | Card radius, shadow                                  | Card spacing | Button text          |
+| ----- | ---------------------------- | -------------- | ---------------------------------------------------- | ------------ | -------------------- |
+| vega  | `h-8` / `h-9` / `h-10`       | `rounded-md`   | `rounded-xl`, `shadow-xs`                            | 6, sm 4      | `text-sm`            |
+| nova  | `h-7` / `h-8` / `h-9`        | `rounded-lg`   | `rounded-xl`                                         | 4, sm 3      | `text-sm`            |
+| maia  | `h-8` / `h-9` / `h-10`       | `rounded-4xl`  | `rounded-2xl`                                        | 6, sm 4      | `text-sm`            |
+| lyra  | `h-7` / `h-8` / `h-9`        | `rounded-none` | `rounded-none`                                       | 4, sm 3      | `text-xs`            |
+| mira  | `h-6` / `h-7` / `h-8`        | `rounded-md`   | `rounded-lg`                                         | 4, sm 3      | `text-xs/relaxed`    |
+| luma  | `h-8` / `h-9` / `h-10`       | `rounded-4xl`  | `rounded-4xl`, `shadow-md`                           | 6, sm 4      | `text-sm`            |
+| sera  | `h-9` / `h-10` / `h-11`      | `rounded-none` | square, `shadow-sm`                                  | 8, sm 5      | `text-xs`, uppercase |
+| rhea  | `h-7` / `h-8` / `h-9`        | `rounded-2xl`  | `rounded-[min(var(--radius-4xl),24px)]`, `shadow-sm` | 5, sm 4      | `text-sm`            |
+
+<!-- gen:style-ladder:end -->
+
+Card spacing is the `--card-spacing` step (`--spacing(n)`) a Card pads with; `Card size="sm"` uses the sm step. The default Button text weight is `font-medium` (sera: `font-semibold`).
+
+## What the style owns
+
+- Never set what the style owns on its parts: radius (except `rounded-full`), height, horizontal padding, shadow, ring or font size. Pick `size` and `variant` instead.
+- A panel is a `Card` or a `Frame`, never a div with `rounded-*`, `border` and `bg-card`; set density with `Card size="sm"` or Frame `spacing`.
+- A bespoke element beside a control (a chip, a custom trigger) takes its height and radius from that control's row above.
+- A focusable control you build copies the focus classes of your installed `input.tsx`.
+- A variant before a class: `className` is for layout (placement, width, gap). `size-*` when width equals height, `cn()` for conditional classes (never a template literal), a scale step before an arbitrary value (`w-60`, not `w-[240px]`), no inline `style` for color or spacing, and no `z-index` on dialogs, sheets or popovers.
+- Change the look through the theme CSS variables in `:root` and `.dark` (`--radius`, `--primary` and the rest) or with a style switch (a new `style` in `components.json`, which means reinstalling the components; ask the user first). Never edit installed files to restyle them.
+
+## Components with no per-style values
+
+`event-calendar`, `gantt`, `icon-stack`, `kanban`, `rating`, `scrollspy`, `sortable` and `timeline` install the same file in every style. Their look comes from your theme tokens and the parts you put inside them (a `Card` in a kanban column follows the style), so never copy another style's radius onto them.

.agents/skills/reui/rules/workflow.md

@@ -2,23 +2,33 @@
 
 The core ReUI loop. The MCP tells you what to install and gives you the API; the shadcn CLI installs it; you turn the installed files into correct, themed, data-wired code by **reuse**, not redesign.
 
+## When to reach for ReUI vs plain shadcn
+
+- A full page or section (dashboard, billing, auth, pricing, settings): `compose_page` first (step 1).
+- A data table with sorting, filtering, pagination or virtualization: the `data-grid` component, never a hand-rolled `<table>`.
+- A drag-and-drop board: the `kanban` component.
+- Column filters, a date range, a tree, a stepper and the like: the matching ReUI component.
+- A single generic control shadcn already has (Button, Dialog, Select): plain shadcn.
+
 ## 1. Find (ReUI MCP `search` / `compose_page`)
 
-**Full multi-section page ask?** Call `compose_page(intent, sections?)` FIRST, before searching block-by-block. It returns ordered sections, each with the best block for the intent (top pick + alternates); on a free account it answers `mode: "free"` and fills the same sections with free `c-*` examples instead, so the plan is always buildable. Sections listed in `unavailableSections` have no real inventory - compose those from components, do not force a bad block.
+**Full multi-section page ask?** Call `compose_page(intent)` FIRST (`sections`, 1 to 10, caps how many sections, default 8), before searching block-by-block. It returns ordered sections, each with the best block for the intent (top pick + alternates); on a free account it answers `mode: "free"` and fills the same sections with free `c-*` examples instead, so the plan is always buildable. Sections listed in `unavailableSections` have no real inventory - compose those from components, do not force a bad block.
 
 For everything else, call `search` with the user's intent. Pass structured hints whenever you can infer them - you are an LLM, so do the parsing the server cannot:
 
-- `type`: `"component"` (one of the 22 building blocks), `"example"` (a c-\* use-case), `"block"` (a full page/section), `"icon"`.
+- `type`: `"component"` (a ReUI component), `"example"` (a c-\* use-case), `"block"` (a full page/section), `"icon"`.
 - `component`: the ReUI component the request implies (`"data-grid"`, `"kanban"`, ...).
-- `category`, `features` (e.g. `["sortable","pagination"]`), `free`.
+- `category`, `free`.
 
-Example: "build a users management page with filters" -> `search({ query: "users management page with filters", type: "block", component: "data-grid", features: ["filters"] })`.
+Example: "build a users management page with filters" -> `search({ query: "users management page with filters", type: "block", component: "data-grid" })` (on a free account use `type: "example"`).
 
 Each result has `install`, `previewUrl`, `docsUrl`, `componentsUsed`, `score`, `termCoverage`, and `whyMatch`. `score` is relative to the top hit (the top is ~100 by construction), not an absolute quality - compare results to each other, and show the user the top options if several score closely; do not silently guess. A low `termCoverage` means a weak match even with a high score - rephrase or widen.
 
+**Pick with the guidance, not the title.** Look-alikes are common (dialog, sheet and drawer; dozens of data-grid examples and app shells). Before installing, read the item's guidance: `get_component` returns `whenToUse` and `whenNotToUse`, `get_example` and `get_block` return `bestFor` (what sets the item apart from its siblings), `agentNotes` and `whenNotToUse`, and a block adds `categoryGuidance` (its category's key note), and `get_examples` rows carry `bestFor`. Each `whenNotToUse` entry names the item to use instead and its `type` (`component` -> `get_component`, `example` -> `get_example`, `block` -> `get_block`, `block-category` "<group>/<category>" -> `search({ query, type: "block", category: "<category>" })`); when the user's situation matches its `when`, switch. Apply every `agentNotes` line before you ship: it names the real identifiers to replace or wire (sample data, stubbed actions, required props). An item with no guidance yet has none of these keys; fall back to its description and thumbnail.
+
 Results are scoped to the account's plan, so you are never shown an item the user could not install: blocks need Pro, Motion Icons need Ultimate. A `type` hint the plan cannot use (or a query whose only matches are premium) comes back as a normal HTTP 200 answer with `locked: true` and `requiredPlan` rather than results - that is a paywall, not a missing item, so switch to free components and `c-*` examples and mention https://reui.io/pricing?ref=skill.
 
-**Always show the preview link.** Whenever you list or recommend items - from `search`, `search_icons`, `list_components`, `compose_page`, or a getter - include each item's `previewUrl` (a live preview page) so the user can SEE it before you install. Blocks and examples link to an individual live preview; icons and components to their live category/component page. This applies to every listing, not only a single pick. Where a result carries `thumbnail` (blocks and `c-*` examples), show the image itself as markdown linked to `previewUrl`, so the user sees the block before you install it.
+**Show the preview.** List every item with its `previewUrl` (a live preview page). Where a result carries `thumbnail` (blocks, `c-*` examples and icons), show the image as markdown linked to `previewUrl`, so the user sees it before you install.
 
 ## 2. Install (shadcn CLI)
 
@@ -40,14 +50,14 @@ Before writing code against any component an item uses:
 
 ## 4. Adapt (reuse-first) - do not skip
 
-Installing files is not the end, and redesigning them defeats the point. First note the project's **base** so you write the right API - read `components.json` -> `style` and take the segment before the first `-` (`base-nova` -> Base UI, `radix-nova` -> Radix UI), see [components.md](./components.md). After `add`:
+Installing files is not the end, and redesigning them defeats the point. Write against your project's base (see [components.md](./components.md)). After `add`:
 
 1. **Read the added files**; keep the composition intact. For a block, verify the components are wired correctly (for `data-grid`: a `useTable({ features: dataGridFeatures, ... })` instance passed as `table`, `recordCount` set - see [components.md](./components.md)).
 2. **Replace demo data with the user's real data** via typed structures (see [adapting.md](./adapting.md)).
 3. **Fix icon imports** to the project's icon library (see [icons.md](./icons.md)).
-4. **Align styling** to semantic tokens and the active theme - no raw colors (see [styling.md](./styling.md)).
+4. **Align styling** to the active theme: semantic tokens for anything that carries meaning, raw palette only for categorical identity and always with its `dark:` pair (see [color.md](./color.md)).
 5. **Validate before finalizing**: if your adaptation introduced components or props you did not read in an `api` or example, run `validate_usage` on them.
-6. **Hit the craft bar** - clear hierarchy, deliberate density, the empty / loading / error states, subtle motion, and mobile-first responsiveness (see [craft.md](./craft.md)). Generic-looking output means you under-reused the design, not that it needs restyling.
+6. **Hit the craft bar** - clear hierarchy, deliberate density, the empty / loading / error states, motion that leaves installed components alone, and mobile-first responsiveness (see [layout.md](./layout.md) and [motion.md](./motion.md)). Generic-looking output means you under-reused the design, not that it needs restyling.
 7. **Pass the quality gates** (security, a11y, scroll) - call the MCP `get_audit_checklist` tool and clear every item (see [quality.md](./quality.md)).
 8. **Typecheck / lint**.
 

.agents/skills/reui/tools.md

@@ -1,6 +1,6 @@
 # ReUI MCP: full reference
 
-The ReUI MCP (`https://mcp.reui.io`, Streamable HTTP) is free to use but needs a ReUI account: on first use the agent signs in with ReUI (a free account is created if the user has none), so every request is tied to an account. Free covers components and examples; a Pro or Ultimate license unlocks premium blocks and Motion Icons and removes the daily request limit. It does **discovery + guidance** (search, inline APIs, page planning, validation) and never serves source; the shadcn CLI does **installation**, and the license key lives there (the `@reui` entry in `components.json`, backed by `.env.local`). Goal: from the user's intent to correct, themed, data-wired ReUI code in the **fewest tokens and calls**, with **no guessing**.
+The ReUI MCP (`https://mcp.reui.io`, Streamable HTTP) is free to use but needs a ReUI account: on first use the agent signs in with ReUI (a free account is created if the user has none), so every request is tied to an account. Free covers components and examples; Pro unlocks premium blocks, Ultimate adds Motion Icons, and either removes the daily request limit by default. It does **discovery + guidance** (search, inline APIs, page planning, validation) and never serves source; the shadcn CLI does **installation**, and the license key lives there (the `@reui` entry in `components.json`, backed by `.env.local`). Goal: from the user's intent to correct, themed, data-wired ReUI code in the **fewest tokens and calls**, with **no guessing**.
 
 ## Golden path (token-optimal - follow this order)
 
@@ -16,21 +16,30 @@ If you already know the exact item name, skip `search`. Everything else is situa
 
 ## The 5 task-specific tools (when to reach for each)
 
-- **`compose_page`** - BEFORE building any full page (dashboard, settings, billing, landing). Pass the intent (and optionally the sections you want); it returns ordered sections, each with the best premium block for the intent (top pick + alternates). On a free account (or in free-only mode) it answers with `mode: "free"` and composes the same sections from free `c-*` examples instead, so the plan is always buildable. Sections with no real inventory are listed honestly in `unavailableSections` - compose those from components instead of forcing a bad block.
+- **`compose_page`** - BEFORE building any full page (dashboard, settings, billing, landing). Pass the intent (`sections` is a number, 1 to 10: the most sections to return, default 8); it returns ordered sections, each with the best premium block for the intent (top pick + alternates). On a free account (or in free-only mode) it answers with `mode: "free"` and composes the same sections from free `c-*` examples instead, so the plan is always buildable. Sections with no real inventory are listed honestly in `unavailableSections` - compose those from components instead of forcing a bad block.
-- **`search_icons`** - whenever you need icons, especially several. Batch up to 24 concepts in one call; each concept returns its best icons with install commands. Pass `animated: true` to get only icons that have a hover-animated Motion variant. Motion Icons are Ultimate for **discovery as well as install**: without an Ultimate credential this returns `locked: true` and zero concepts, so check the plan before promising the user icons.
+- **`search_icons`** - whenever you need icons, especially several. Batch up to 24 concepts in one call; each concept returns its best icons with install commands. Only some icons have a real hover animation: `animated: true` returns only those, and a concept with none comes back empty with an `animatedNote`. Each icon carries a `thumbnail` of its 4 styles; look at it before trusting a name. An old icon name still resolves (`renamedFrom` names it), unless it is also an ordinary word icons are named for today (`watch`): then it ranks today's icons for the word and `oldNameNotes` names the icon the old name installs. Motion Icons are Ultimate for **discovery as well as install**: without an Ultimate credential this returns `locked: true` and zero concepts, so check the plan before promising the user icons.
 - **`validate_usage`** - BEFORE writing code with component names or props you have not read in an inline `api` or an installed example. It checks planned names + props against the indexed API docs and registry item names; returns did-you-mean suggestions and per-prop documented / notDocumented verdicts. Deterministic, no inference - a notDocumented prop means stop and read the API, not push on.
 - **`get_thumbnail`** - when the choice is VISUAL and the ranking has not already made it: 2 or more candidates within about 15 score points. When the top hit leads by a wide margin, or `missedTerms` already rules the others out, skip it; an image costs roughly 10 times a result row. It returns the hosted preview image as image content so you can look yourself; pass up to 4 names in one call to compare `compose_page` alternates side by side before installing any of them. To let the USER see something, share the item's `thumbnail` and `previewUrl` instead - this tool is for your own eyes, and a client that cannot render images gets the URLs in the text block.
-- **`get_agent_skill`** - when the agent has no local ReUI skill (a cloud or tools-only client), or to check whether the installed one is stale: it returns the current workflow inline plus the installer command, and its `version` can be compared with the version stamped at the top of your local `SKILL.md`.
+- **`get_agent_skill`** - when the agent has no local ReUI skill (a cloud or tools-only client), or to check whether the installed one is stale. With no `topic` it returns the core workflow, the topic index and the installer; `topic` (for example `motion`) returns that one file in full. Compare its `version` with the one stamped atop your local `SKILL.md`.
 
 ## All 19 tools
 
 `search`, `get_block`, `get_example`, `get_icon`, `get_thumbnail`, `list_block_groups`, `list_block_categories`, `list_example_categories`, `list_icon_categories`, `list_components`, `get_component`, `get_examples`, `search_icons`, `compose_page`, `validate_usage`, `get_install_command`, `get_project_context`, `get_agent_skill`, `get_audit_checklist`. The MCP serves the full parameter schemas; do not guess parameters beyond them.
 
+## Commands (MCP prompts)
+
+In Claude Code the ReUI MCP adds these prompts; other agents show MCP prompts under their own names, or not at all. Plain language runs the same loop.
+
+- `/mcp__reui__build <what>`: compose a page, section or feature (plan, install, read the API, adapt, audit).
+- `/mcp__reui__add <item>`: find, install and wire in 1 component, example, block or icon.
+- `/mcp__reui__fix [target]`: fix ReUI usage (undocumented props, a base or radix mismatch, missing states, accessibility, scroll).
+- `/mcp__reui__improve [target]`: refine existing ReUI UI (hierarchy, density, states, responsive, motion).
+
 ## Token + speed rules
 
 - **Batch `get_component`** - ONE call with the whole `componentsUsed` array, never N calls. Skip it entirely when `componentDigests` already answers the question.
 - **Read source by installing** - the MCP serves no source. To read or analyze an item's real code, install it with the shadcn CLI and open the local files. Learn an API from the inline `api` / `componentDigests`, never by reading raw source.
-- **Infer `search` hints yourself** (`type`, `component`, `category`, `features`, `free`) - hints shrink the result set and the tokens. Keep `limit` low; one right result beats ten.
+- **Infer `search` hints yourself** (`type`, `component`, `category`, `free`) - hints shrink the result set and the tokens. Keep `limit` low; one right result beats ten.
 - Run independent calls (and the shadcn install) concurrently in one turn - serial tool calls are the main source of slowness.
 - Don't repeat a search for the same intent; don't call `list_*` to "see everything" - `search` is the entry point, `list_*` is only for browsing a taxonomy the user explicitly wants to explore.
 - Prefer `get_component`'s inline `api` over `docsUrl` / `/llms.txt`. Fetch a web page only as a last resort.
@@ -42,11 +51,12 @@ If you already know the exact item name, skip `search`. Everything else is situa
 - Each result carries `whyMatch`, `install`, docs/preview URLs, and a `free` flag; premium items carry `requiredPlan` (`"pro"` for blocks, `"ultimate"` for icons).
 - `componentDigests` is a top-level map: a compact API contract per referenced component - often enough to wire an item without a `get_component` call.
 - An inline `api` is not always the whole API: a very large one is trimmed on heading boundaries to fit your context, and the response names every dropped heading in `sectionsOmitted` plus a `next` hint for re-reading one. Never treat a trimmed capsule as complete; `validate_usage` still checks the FULL API.
-- Icon results and `get_icon` include `animated: true` and `installAnimated` when a hover-animated Motion variant exists (animated: `@reui/icons/animated/<style>/<name>`; static: `@reui/icons/default/<style>/<name>`).
+- Icon results and `get_icon` include `animated: true` and `installAnimated` only when the icon has a real hover animation (animated: `@reui/icons/animated/<style>/<name>`; static: `@reui/icons/default/<style>/<name>`). Use the strings given; never build an animated path by hand, it installs a static copy for every other icon.
-- Blocks and `c-*` examples carry `thumbnail`, an absolute URL to a hosted preview image (3:2; blocks 1080x720, examples 900x600, light theme). Show the top 2 or 3 as markdown images linked to `previewUrl` so the user picks by eye; use `get_thumbnail` when YOU need to see them. Components and icons have no thumbnail.
+- Blocks, `c-*` examples and icons carry `thumbnail`, an absolute URL to a hosted preview image (light theme; blocks 1080x720, examples 900x600, icons 640x200 with the 4 styles side by side). Show the top 2 or 3 as markdown images linked to `previewUrl` so the user picks by eye; use `get_thumbnail` when YOU need to see them. Components have no thumbnail.
 - Each result carries `missedTerms`, the words from YOUR query it does not mention (omitted when it matches them all). A high score with 3 missed words is a near miss wearing a good score: read this before `get_thumbnail`, it is far cheaper.
 - `weakMatch: true` means the top result misses more of the query's words than it matches; `unmatchedTerms` lists the words none of the top results mention and `weakMatchNote` says what to do. Stop and re-search with the registry's own words, or compose from components, instead of installing the top hit.
 - On a free account `premiumPicks` (a sibling of `results`, never inside it) lists up to 3 premium blocks that fit the query more closely than the free answer, as previews with no install command, plus `unlock` with the plan and a pricing link. Mention the upgrade at most once per conversation, then keep building with what the plan covers. `weakMatch` and `premiumPicks` can appear together: the free answer is weak AND a premium block fits; `weakMatchNote` says so.
+- Guidance is getter-only, never on search rows: `get_component` adds `whenToUse`, `whenNotToUse` and `agentNotes`; `get_example` and `get_block` add `bestFor`, `agentNotes` and `whenNotToUse`, and `get_block` also `categoryGuidance`; `get_examples` rows add `bestFor`. Answers carry the key lines; the item page's `.md` twin has every line. Each `whenNotToUse` entry is `{ when, instead, type }`: fetch `instead` with the getter its `type` names.
 - Results are scoped to the caller's plan, so a free account never sees a block or icon it could not install. When a search matches only hidden premium items, the answer says so (`locked`, `requiredPlan`, `premiumMatches`) instead of pretending the registry is empty.
 
 ## Error playbook

.gitea/workflows/cd.yaml

@@ -7,6 +7,10 @@ on:
 permissions:
   contents: read
 
+concurrency:
+  group: evofw-release
+  cancel-in-progress: false
+
 jobs:
   quality:
     uses: ./.gitea/workflows/quality.yaml
@@ -96,9 +100,13 @@ jobs:
         if: steps.rel.outputs.released == 'true'
         uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
         with:
-          name: evofw
+          name: evofw-${{ github.run_id }}-${{ github.run_attempt }}-publish
           driver: docker-container
-          cleanup: false
+          cleanup: true
+          buildkitd-config: deploy/docker/buildkitd.toml
+      - name: Verify run-scoped builder effective GC configuration
+        if: steps.rel.outputs.released == 'true'
+        run: sh scripts/ci/prepare-builder.sh evofw-${{ github.run_id }}-${{ github.run_attempt }}-publish deploy/docker/buildkitd.toml
       - name: Prepare image metadata
         if: steps.rel.outputs.released == 'true'
         id: meta

.gitea/workflows/quality.yaml

@@ -116,8 +116,9 @@ jobs:
               .gitea/workflows/*|scripts/*)
                 full_pipeline=true
                 ;;
-              docs/openapi.yaml|redocly.yaml)
+              docs/openapi.yaml|docs/openapi.schemas.json|redocly.yaml)
                 openapi=true
+                api=true
                 ;;
               .cursor/*|.claude/*|.codegraph/*|.agents/*)
                 ;;
@@ -270,7 +271,13 @@ jobs:
           # turbo builds @evofw/{shared,db} first — api typecheck resolves their dist types
           pnpm exec turbo run typecheck --filter=@evofw/api --filter=@evofw/db --filter=@evofw/shared
           pnpm exec turbo run test --filter=@evofw/api --filter=@evofw/shared
+          node --test scripts/ci/sqlite-backup.test.mjs
+          node --test scripts/ci/image-smoke-marker.test.mjs
+          sh scripts/ci/restore-wrapper.test.sh
           pnpm exec turbo run build --filter=@evofw/api
+          node --test scripts/ci/benchmark-fixture.test.mjs
+          node --test scripts/ci/manifest-platform.test.mjs
+          node scripts/ci/generate-openapi-schemas.mjs --check
 
   commitlint:
     if: inputs.is_pull_request
@@ -311,17 +318,38 @@ jobs:
           sh scripts/ci/pnpm-ci.sh
           pnpm exec commitlint --from "$BASE_SHA" --to "$HEAD_SHA"
 
-  docker-check:
+  linux-agent-smoke:
     needs: [changes]
-    if: inputs.is_pull_request && needs.changes.outputs.docker == 'true'
+    if: needs.changes.outputs.api == 'true' || needs.changes.outputs.docker == 'true'
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - name: Run shipped agent in a dedicated namespace-capable helper
+        timeout-minutes: 10
+        run: |
+          sh scripts/ci/linux-agent-container-smoke.test.sh
+          sh scripts/ci/linux-agent-container-smoke.sh
+
+  docker-check:
+    needs: [changes]
+    # Every CD invocation must pass upgrade validation, including docs-only releases.
+    if: "!inputs.is_pull_request || needs.changes.outputs.docker == 'true' || needs.changes.outputs.api == 'true' || needs.changes.outputs.web == 'true'"
+    runs-on: ubuntu-latest
+    env:
+      SMOKE_IMAGE: evofw-smoke:${{ github.run_id }}-${{ github.run_attempt }}
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+        with:
+          node-version: "22"
       - uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
         with:
-          name: evofw
+          name: evofw-${{ github.run_id }}-${{ github.run_attempt }}-quality
           driver: docker-container
-          cleanup: false
+          cleanup: true
+          buildkitd-config: deploy/docker/buildkitd.toml
+      - name: Verify run-scoped builder effective GC configuration
+        run: sh scripts/ci/prepare-builder.sh evofw-${{ github.run_id }}-${{ github.run_attempt }}-quality deploy/docker/buildkitd.toml
       - name: Log in to Gitea Registry
         if: inputs.allow_registry_login
         uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
@@ -335,8 +363,7 @@ jobs:
           BUILDX_BAKE_ENTITLEMENTS_FS: "0"
           BUILDX_BAKE_FILE_RELATIVE_PATHS: "1"
         run: docker buildx bake --allow=fs.read="${{ github.workspace }}" -f docker-bake.hcl --print default
-      - name: bake (no push)
+      - name: Build smoke image (read release cache, no cache publication)
-        if: inputs.allow_registry_login
         working-directory: deploy/docker
         env:
           BUILDX_BAKE_ENTITLEMENTS_FS: "0"
@@ -345,4 +372,42 @@ jobs:
           set -euxo pipefail
           owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
           export CACHE_REF_NODE="git.shx.one/${owner_lc}/evofw-buildcache:node-buildcache"
-          docker buildx bake --allow=fs.read="${{ github.workspace }}" -f docker-bake.hcl default
+          docker buildx bake --allow=fs.read="${{ github.workspace }}" -f docker-bake.hcl \
+            --set evofw.cache-to= --set "evofw.tags=$SMOKE_IMAGE" --load default
+      - name: Capture exact built image identity
+        id: built-image
+        run: echo "id=$(docker image inspect --format '{{.Id}}' "$SMOKE_IMAGE")" >> "$GITHUB_OUTPUT"
+      - name: Verify upgrade baseline resolver
+        run: sh scripts/ci/resolve-upgrade-baseline.test.sh
+      - name: Resolve immutable upgrade baseline
+        id: upgrade-baseline
+        env:
+          PREVIOUS_IMAGE: ${{ vars.EVOFW_PREVIOUS_IMAGE }}
+          REQUIRE_UPGRADE: ${{ !inputs.is_pull_request }}
+          FIRST_RELEASE: ${{ vars.EVOFW_FIRST_RELEASE == 'true' }}
+          REPOSITORY_OWNER: ${{ github.repository_owner }}
+        run: |
+          set -eu
+          owner_lc="$(printf '%s' "$REPOSITORY_OWNER" | tr '[:upper:]' '[:lower:]')"
+          export IMAGE_REPOSITORY="git.shx.one/$owner_lc/evofw"
+          baseline="$(sh scripts/ci/resolve-upgrade-baseline.sh)"
+          printf 'image=%s\n' "$baseline" >> "$GITHUB_OUTPUT"
+      - name: Built image runtime smoke
+        env:
+          PREVIOUS_IMAGE: ${{ steps.upgrade-baseline.outputs.image }}
+          REQUIRE_UPGRADE: ${{ !inputs.is_pull_request }}
+          FIRST_RELEASE: ${{ vars.EVOFW_FIRST_RELEASE == 'true' }}
+          SMOKE_ARTIFACT_DIR: runtime-artifacts
+          BUILT_IMAGE: ${{ steps.built-image.outputs.id }}
+        run: sh scripts/ci/image-smoke.sh "$BUILT_IMAGE" "${PREVIOUS_IMAGE:-}"
+      - name: Built image constrained steady performance and correctness gate
+        timeout-minutes: 15
+        env:
+          BUILT_IMAGE: ${{ steps.built-image.outputs.id }}
+          BENCH_P95_LIMIT_MS: "500"
+          BENCH_CONCURRENCY: "10"
+          BENCH_STEADY_CONCURRENCY: "2"
+          BENCH_STEADY_INTERVAL_MS: "1200"
+        run: |
+          sh scripts/bench/runtime.test.sh
+          sh scripts/bench/runtime.sh "$BUILT_IMAGE" "runtime-artifacts/benchmark-${{ github.run_id }}-${{ github.run_attempt }}"

.serena/.gitignore

@@ -0,0 +1,2 @@
+/cache
+/project.local.yml

.serena/project.yml

@@ -0,0 +1,187 @@
+# the name by which the project can be referenced within Serena/when chatting with the LLM.
+project_name: "EvoFirewall"
+
+# list of language servers to start when using the LSP backend; choose from:
+#   ada                    al                     angular                ansible                astro
+#   bash                   bsl                    clojure                cpp                    cpp_ccls
+#   crystal                csharp                 csharp_omnisharp       cue                    dart
+#   deno                   elixir                 elm                    erlang                 fortran
+#   fsharp                 gdscript               gleam                  go                     groovy
+#   haskell                haxe                   hlsl                   html                   java
+#   json                   julia                  julia_fatou            kotlin                 latex
+#   lean4                  lua                    lua_emmylua            luau                   markdown
+#   matlab                 msl                    nextflow               nix                    ocaml
+#   pascal                 perl                   php                    php_devsense           php_phpactor
+#   php_phpantom           powershell             python                 python_basedpyright    python_jedi
+#   python_pyrefly         python_ty              qml                    r                      rego
+#   ruby                   ruby_solargraph        rust                   scala                  scss
+#   solidity               svelte                 swift                  systemverilog          terraform
+#   toml                   typescript             typescript_tsgo        typescript_vts         vue
+#   vbnet                  wolfram                yaml                   zig
+#   (This list may be outdated; generated with scripts/print_language_list.py;
+#   For the current list, see values of the LanguageServerId enum here:
+#   https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
+# For some languages, there are several alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
+# Note:
+#   - For C, use cpp
+#   - For JavaScript, use typescript
+#   - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
+#   - For Astro projects, use astro (subsumes typescript/javascript for .astro projects; requires npm)
+#   - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
+#   - For Deno projects, use deno (serves the same .ts/.js files as typescript; requires the deno CLI on PATH)
+#   - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
+#   - For Free Pascal/Lazarus, use pascal
+#   - External Python adapters may add further registered IDs; install the adapter package first
+#     and then use its ID here, for example: example
+# Special requirements:
+#   Some language servers require additional setup/installations.
+#   See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
+# When using multiple language servers, the first language server that supports a given file will be used for that file.
+# The first language server is the default language and the respective language server will be used as a fallback.
+# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
+language_servers:
+- typescript
+
+# the encoding used by text files in the project
+# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
+encoding: "utf-8"
+
+# optional shell command to run before the language backend (LSP or JetBrains) is initialised.
+# the command runs in the project root directory and is only executed if the project is trusted
+# (see trusted_project_path_patterns in the global configuration).
+# serena waits for the command to exit: a non-zero exit code is logged as an error but does not
+# abort activation. a per-project timeout (activation_command_timeout, default 180s) is the safety
+# backstop for non-terminating commands; on expiry the process is killed and activation continues.
+# example: activation_command: "npx nx run-many -t build"
+activation_command:
+
+# maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
+# must be a positive number.
+activation_command_timeout: 180.0
+
+# line ending convention to use when writing source files.
+# Possible values: unset (use global setting), "lf", "crlf", or "native" (platform default)
+# This does not affect Serena's own files (e.g. memories and configuration files), which always use native line endings.
+line_ending:
+
+# The language backend to use for this project.
+# If not set, the global setting from serena_config.yml is used.
+# Valid values: LSP, JetBrains
+# Note: the backend is fixed at startup. If a project with a different backend
+# is activated post-init, an error will be returned.
+language_backend:
+
+# The interface through which the agent (LLM) accesses Serena's functionality (overrides the global setting).
+# Valid values: tools, REPL (see the global configuration for details); leave empty to use the global setting.
+# Note: the interface is fixed at startup. If a project is activated post-init, its setting is not applied.
+agent_interface:
+
+# whether to use project's .gitignore files to ignore files
+ignore_all_files_in_gitignore: true
+
+# advanced configuration option allowing to configure language server-specific options.
+# Maps the language key to the options.
+# The settings are considered only if the project is trusted (see global configuration to define trusted projects).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#language-server-specific-settings
+ls_specific_settings: {}
+
+# list of workspace folder paths (LSP backend only).
+# These folders will be used to build up Serena's symbol index.
+# Paths must be within the project root and should thus be relative to the project root.
+# Furthermore, the paths should not be filtered by ignore settings.
+# Default setting: The entire project root folder (".") is considered.
+# In (large) monorepos, this can be used to index only subfolders of the project root, e.g.
+#   ls_workspace_folders:
+#     - "./subproject1"
+#     - "./subproject2"
+ls_workspace_folders:
+- "."
+
+# list of additional workspace folder paths for cross-package reference support.
+# Paths can be absolute or relative to the project root.
+# Each folder is registered as an LSP workspace folder, enabling language servers to discover
+# symbols and references across package boundaries, but these folders are not indexed by Serena,
+# i.e. the respective symbols will not be found using Serena's symbol search tools.
+# Example:
+#   additional_workspace_folders:
+#     - ../sibling-package
+#     - ../shared-lib
+ls_additional_workspace_folders: []
+
+# list of additional paths to ignore in this project.
+# Same syntax as gitignore, so you can use * and **.
+# Important: quote patterns that start with `*`, otherwise YAML treats them as aliases.
+# Example:
+#   ignored_paths:
+#     - "examples/**"
+#     - ".worktrees/**"
+#     - "**/bin/**"
+#     - "**/obj/**"
+# Note: global ignored_paths from serena_config.yml are also applied additively.
+ignored_paths: []
+
+# whether the project is in read-only mode
+# If set to true, all editing tools will be disabled and attempts to use them will result in an error
+# Added on 2025-04-18
+read_only: false
+
+# list of tool names to exclude.
+# This extends the existing exclusions (e.g. from the global configuration)
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+excluded_tools: []
+
+# list of tools to include that would otherwise be disabled (particularly optional tools that are disabled by default).
+# This extends the existing inclusions (e.g. from the global configuration).
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+included_optional_tools: []
+
+# fixed set of tools to use as the base tool set (if non-empty), replacing Serena's default set of tools.
+# This cannot be combined with non-empty excluded_tools or included_optional_tools.
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+fixed_tools: []
+
+# list of APIs (facades or facade methods, e.g. "lsp" or "lsp.find_symbol") to exclude from the REPL.
+# This extends the existing exclusions (e.g. from the global configuration).
+excluded_apis: []
+
+# list of APIs (facades or facade methods, e.g. "lsp" or "lsp.get_diagnostics_for_symbol") to include in the REPL
+# that would otherwise be disabled (particularly optional methods, which are disabled by default).
+# This extends the existing inclusions (e.g. from the global configuration).
+included_apis: []
+
+# list of mode names that are to be activated by default, overriding the setting in the global configuration.
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# If the setting is undefined/empty, the default_modes from the global configuration (serena_config.yml) apply.
+# Otherwise, this overrides the setting from the global configuration (serena_config.yml).
+# Therefore, you can set this to [] if you do not want the default modes defined in the global config to apply
+# for this project.
+# This setting can, in turn, be overridden by CLI parameters (--mode).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+default_modes:
+
+# list of mode names to be activated additionally for this project, e.g. ["query-projects"]
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+added_modes:
+
+# initial prompt for the project. It will always be given to the LLM upon activating the project
+# (contrary to the memories, which are loaded on demand).
+initial_prompt: ""
+
+# time budget (seconds) per tool call for the retrieval of additional symbol information
+# such as docstrings or parameter information.
+# This overrides the corresponding setting in the global configuration; see the documentation there.
+# If null or missing, use the setting from the global configuration.
+symbol_info_budget:
+
+# list of regex patterns which, when matched, mark a memory entry as read‑only.
+# Extends the list from the global configuration, merging the two lists.
+read_only_memory_patterns: []
+
+# list of regex patterns for memories to completely ignore.
+# Matching memories will not appear in list_memories or activate_project output
+# and cannot be accessed via read_memory or write_memory.
+# To access ignored memory files, use the read_file tool on the raw file path.
+# Extends the list from the global configuration, merging the two lists.
+# Example: ["_archive/.*", "_episodes/.*"]
+ignored_memory_patterns: []

AGENTS.md

@@ -28,7 +28,7 @@ Surface: **frame**. Hierarchy: **ReUI PRO > shadcn**. Contract: `docs/ui-design-
 MCP `reui` (PRO, Bearer `REUI_LICENSE_KEY`) primary + `shadcn` primitives — локально: `.zcode/config.json` (ZCode, шаблон `.zcode/config.json.example`) / `.cursor/mcp.json` (Cursor). CLI из `apps/web`.  
 KPI/QuickActions: EvoBGP hybrid SoT (`KpiStatGrid`, `QuickActionGrid`).  
 Docs: [Introduction](https://reui.io/docs) · [llms.txt](https://reui.io/llms.txt) · [MCP](https://reui.io/docs/mcp) · [Agent Skills](https://reui.io/docs/agent-skills).  
-Skill: `.agents/skills/reui` (v `d9f4a302f4`); обновление: `curl.exe -fsSL https://mcp.reui.io/install | node -` из корня проекта.
+Skill: `.agents/skills/reui` (v `68f0e6fa91`); обновление: `curl.exe -fsSL https://mcp.reui.io/install | node -` из корня проекта.
 
 ## Docs
 

apps/api/package.json

@@ -6,7 +6,7 @@
   "scripts": {
     "dev": "tsx watch src/server.ts",
     "typecheck": "tsc --noEmit",
-    "build": "tsup src/server.ts --format esm --dts --publicDir src/agent-scripts && node -e \"const fs=require('fs');const p='dist/agent-scripts';fs.mkdirSync(p,{recursive:true});for(const f of fs.readdirSync('src/agent-scripts'))fs.copyFileSync('src/agent-scripts/'+f,p+'/'+f)\"",
+    "build": "tsup src/server.ts --format esm --dts && node -e \"const fs=require('fs');const p='dist/agent-scripts';fs.mkdirSync(p,{recursive:true});for(const f of fs.readdirSync('src/agent-scripts'))fs.copyFileSync('src/agent-scripts/'+f,p+'/'+f)\"",
     "start": "node dist/server.js",
     "test": "vitest run"
   },

apps/api/src/agent-scripts/install.sh

@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 # EvoFirewall Linux install one-liner
 # Re-run on an already-installed host updates scripts/timer **and forces nft
-# rule re-apply** (clears last_hash), keeping credentials
+# rule re-apply** (clears kernel verification), keeping credentials
 # (unless EVOFW_INSTALL_FORCE=1 → full re-enroll).
 set -euo pipefail
 
@@ -49,13 +49,12 @@ ensure_pkg() {
 ensure_pkg curl curl || exit 1
 need_cmd bash || { echo "missing bash" >&2; exit 1; }
 
-# JSON parse for enroll; python3 is an acceptable fallback in sync script.
+# Policy validation and nft batch generation require Python; jq alone is not
-if ! need_cmd jq && ! need_cmd python3; then
+# sufficient. util-linux supplies flock for serialized timer/manual sync runs.
-  ensure_pkg jq jq || ensure_pkg python3 python3 || {
+ensure_pkg python3 python3 || exit 1
-    echo "evofw install: need jq or python3" >&2
+ensure_pkg flock util-linux || exit 1
-    exit 1
+ensure_pkg timeout coreutils || exit 1
-  }
+
-fi
 
 # Firewall tools: never reinstall/replace if already present.
 if need_cmd nft; then
@@ -63,8 +62,8 @@ if need_cmd nft; then
 elif need_cmd iptables; then
   echo "evofw install: iptables already present — skip nftables/iptables install"
   if ! need_cmd ipset; then
-    echo "evofw install: installing ipset (optional companion for iptables)..."
+    echo "evofw install: installing ipset (required companion for iptables)..."
-    pkg_install ipset 2>/dev/null || true
+    ensure_pkg ipset ipset || exit 1
   fi
 else
   echo "evofw install: no nft/iptables — installing firewall backend..."
@@ -111,7 +110,7 @@ if [[ "$HAS_SYSTEMD" -ne 1 ]]; then
 fi
 
 : "${EVOFW_CP_URL:?EVOFW_CP_URL required}"
-: "${EVOFW_SEED:?EVOFW_SEED required}"
+: "${EVOFW_ENROLLMENT_CREDENTIAL:?Scoped EVOFW_ENROLLMENT_CREDENTIAL required}"
 : "${EVOFW_CLIENT_NAME:?EVOFW_CLIENT_NAME required}"
 
 CONF_DIR=/etc/evofw
@@ -161,14 +160,24 @@ write_conf() {
     printf 'CLIENT_TOKEN=%s\n' "$(shell_quote "$client_token")"
     printf 'CLIENT_NAME=%s\n' "$(shell_quote "$client_name")"
     printf 'KERNEL_BACKEND=%s\n' "$(shell_quote "$backend")"
-  } >"$CONF_FILE"
+  } >"${CONF_FILE}.new"
-  chmod 600 "$CONF_FILE"
+  chmod 600 "${CONF_FILE}.new"
+  python3 - "${CONF_FILE}.new" "$CONF_FILE" <<'PY_CONF_COMMIT'
+import os, sys
+with open(sys.argv[1], 'rb') as value: os.fsync(value.fileno())
+os.replace(sys.argv[1], sys.argv[2])
+fd = os.open(os.path.dirname(sys.argv[2]), os.O_RDONLY)
+try: os.fsync(fd)
+finally: os.close(fd)
+PY_CONF_COMMIT
 }
 
 install_sync_and_uninstall() {
   local sync_tmp=$1
-  install -m 755 "$sync_tmp" "$SYNC_SCRIPT"
+  bash -n "$sync_tmp" || return 1
-  if curl -fsSL "${CP_URL}/v1/agent/uninstall.sh" -o "$UNINSTALL_SCRIPT" 2>/dev/null; then
+  install -m 755 "$sync_tmp" "${SYNC_SCRIPT}.new"
+  mv -f "${SYNC_SCRIPT}.new" "$SYNC_SCRIPT"
+  if curl --connect-timeout 10 --max-time 60 -fsSL "${CP_URL}/v1/agent/uninstall.sh" -o "$UNINSTALL_SCRIPT" 2>/dev/null; then
     chmod 755 "$UNINSTALL_SCRIPT"
   else
     echo "evofw install: warning — could not download uninstall.sh (optional)" >&2
@@ -214,7 +223,7 @@ UNIT
 
 download_sync_script() {
   local out=$1
-  if ! curl -fsSL "${CP_URL}/v1/agent/sync-script" -o "$out"; then
+  if ! curl --connect-timeout 10 --max-time 60 -fsSL "${CP_URL}/v1/agent/sync-script" -o "$out"; then
     echo "failed to download sync script from ${CP_URL}/v1/agent/sync-script" >&2
     return 1
   fi
@@ -222,6 +231,7 @@ download_sync_script() {
     echo "sync script is not a shell script (CP returned unexpected body)" >&2
     return 1
   fi
+  bash -n "$out"
 }
 
 SYNC_TMP=$(mktemp)
@@ -251,33 +261,56 @@ if [[ -f "$CONF_FILE" && "${EVOFW_INSTALL_FORCE:-}" != "1" ]]; then
   download_sync_script "$SYNC_TMP" || exit 1
   write_conf "$CLIENT_ID" "$CLIENT_TOKEN" "$CLIENT_NAME" "$BACKEND"
   install_sync_and_uninstall "$SYNC_TMP"
-  # Force one apply after script refresh (nft set upgrade, counters, port hits, etc.).
+  # Invalidate structural verification to force reconciliation after refresh.
-  rm -f /var/lib/evofw/last_hash
+  rm -f /var/lib/evofw/kernel_state
   enable_scheduler_and_run
   echo "Updated. Client id=${CLIENT_ID}. Sync script + timer refreshed; nft rules re-applied."
   echo "Force sync: $SYNC_SCRIPT"
-  echo "Uninstall: $UNINSTALL_SCRIPT  (or: curl -fsSL ${CP_URL}/v1/agent/uninstall.sh | bash)"
+  echo "Uninstall: $UNINSTALL_SCRIPT  (or: curl --connect-timeout 10 --max-time 60 -fsSL ${CP_URL}/v1/agent/uninstall.sh | bash)"
   exit 0
 fi
 
 # --- Fresh install (or EVOFW_INSTALL_FORCE=1) ---
-CLIENT_TOKEN="$(gen_token)"
+mkdir -p "$CONF_DIR" /var/lib/evofw
+chmod 700 "$CONF_DIR"
+exec 8>/var/lib/evofw/sync.lock
+flock -x 8
+ENROLL_PENDING="$CONF_DIR/enrollment-pending.json"
+CLIENT_TOKEN=$(python3 - "$ENROLL_PENDING" "$CP_URL" "${EVOFW_INSTALL_LINK_ID:-}" <<'PY_ENROLL_PENDING'
+import json, os, secrets, sys
+from pathlib import Path
+path = Path(sys.argv[1]); origin, link = sys.argv[2:]
+if not link: raise SystemExit('install link required')
+if path.exists():
+    state = json.loads(path.read_text())
+    if state.get('origin') != origin or state.get('link') != link: raise SystemExit('pending enrollment belongs to another link')
+else:
+    state = {'origin': origin, 'link': link, 'token': 'evofw_' + secrets.token_urlsafe(32)}
+    fd = os.open(str(path) + '.new', os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
+    with os.fdopen(fd, 'w') as out:
+        json.dump(state, out); out.flush(); os.fsync(out.fileno())
+    os.replace(str(path) + '.new', path)
+    fd = os.open(str(path.parent), os.O_RDONLY)
+    try: os.fsync(fd)
+    finally: os.close(fd)
+print(state['token'])
+PY_ENROLL_PENDING
+)
 HOSTNAME="$(hostname -f 2>/dev/null || hostname)"
 
 # Fail fast: pull sync script before enroll so we never leave a DB agent without a local agent.
 download_sync_script "$SYNC_TMP" || exit 1
 
-if [[ -n "${EVOFW_INSTALL_LINK_ID:-}" ]]; then
+ENROLL_BODY=$(python3 - "$EVOFW_CLIENT_NAME" "$HOSTNAME" "$PLATFORM" "$CLIENT_TOKEN" "$EVOFW_INSTALL_LINK_ID" <<'PY_ENROLL_BODY'
-  ENROLL_BODY=$(printf '{"name":"%s","hostname":"%s","platform":"%s","token":"%s","client_version":"install.sh/1","install_link_id":"%s"}' \
+import json, sys
-    "$EVOFW_CLIENT_NAME" "$HOSTNAME" "$PLATFORM" "$CLIENT_TOKEN" "$EVOFW_INSTALL_LINK_ID")
+name, hostname, platform, token, link = sys.argv[1:]
-else
+print(json.dumps(dict(name=name, hostname=hostname, platform=platform, token=token, client_version='install.sh/2', install_link_id=link)))
-  ENROLL_BODY=$(printf '{"name":"%s","hostname":"%s","platform":"%s","token":"%s","client_version":"install.sh/1"}' \
+PY_ENROLL_BODY
-    "$EVOFW_CLIENT_NAME" "$HOSTNAME" "$PLATFORM" "$CLIENT_TOKEN")
+)
-fi
 
-ENROLL_CODE=$(curl -sS -o "$ENROLL_TMP" -w "%{http_code}" -X POST "${CP_URL}/v1/agent/enroll" \
+ENROLL_CODE=$(curl --connect-timeout 10 --max-time 60 -sS -o "$ENROLL_TMP" -w "%{http_code}" -X POST "${CP_URL}/v1/agent/enroll" \
   -H "Content-Type: application/json" \
-  -H "X-EvoFW-Seed: ${EVOFW_SEED}" \
+  -H "X-EvoFW-Enrollment-Credential: ${EVOFW_ENROLLMENT_CREDENTIAL}" \
   -d "$ENROLL_BODY")
 if [[ "$ENROLL_CODE" != "201" && "$ENROLL_CODE" != "200" ]]; then
   echo "enroll failed: HTTP ${ENROLL_CODE}" >&2
@@ -304,10 +337,14 @@ if [[ -z "$BACKEND" ]]; then
 fi
 
 write_conf "$CLIENT_ID" "$CLIENT_TOKEN" "$EVOFW_CLIENT_NAME" "$BACKEND"
+# Config is committed before pending enrollment can be removed.
+rm -f "$ENROLL_PENDING"
+flock -u 8
+exec 8>&-
 install_sync_and_uninstall "$SYNC_TMP"
 enable_scheduler_and_run
 
 echo "Installed. Client id=${CLIENT_ID}. Approve in EvoFirewall UI (rules optional — can assign later)."
 echo "If still offline after Approve, run: $SYNC_SCRIPT"
 echo "Re-run the same install URL to update scripts without re-enroll."
-echo "Uninstall: $UNINSTALL_SCRIPT  (or: curl -fsSL ${CP_URL}/v1/agent/uninstall.sh | bash)"
+echo "Uninstall: $UNINSTALL_SCRIPT  (or: curl --connect-timeout 10 --max-time 60 -fsSL ${CP_URL}/v1/agent/uninstall.sh | bash)"

apps/api/src/agent-scripts/mikrotik-install.rsc

@@ -1,18 +1,18 @@
 # EvoFirewall MikroTik install (RouterOS 7.21+)
-# Short-link sets EvofwCpUrl / EvofwSeed / EvofwName / EvofwInstallLinkId before body.
+# Short-link sets EvofwCpUrl / EvofwEnrollmentCredential / EvofwName / EvofwInstallLinkId before body.
 #
 # One script only: evofw-sync (credentials + sync). Legacy evofw-env is removed.
-# Sync: GET /v1/agent/policy (JSON) → rebuild address-lists. LastHash in RAM (full rebuild after reboot).
+# Sync: bounded JSON policy -> staged canonical owned rules. Rebuild after reboot.
 
 :global EvofwCpUrl
-:global EvofwSeed
+:global EvofwEnrollmentCredential
 :global EvofwName
 :global EvofwInstallLinkId
 :global EvofwToken
 :global EvofwLastHash
 
 :if ([:typeof $EvofwCpUrl] = "nothing" || [:len $EvofwCpUrl] = 0) do={ :error "EvofwCpUrl required" }
-:if ([:typeof $EvofwSeed] = "nothing" || [:len $EvofwSeed] = 0) do={ :error "EvofwSeed required" }
+:if ([:typeof $EvofwEnrollmentCredential] = "nothing" || [:len $EvofwEnrollmentCredential] = 0) do={ :error "EvofwEnrollmentCredential required" }
 :if ([:typeof $EvofwName] = "nothing" || [:len $EvofwName] = 0) do={ :set EvofwName [/system identity get name] }
 
 :local already 0
@@ -43,61 +43,74 @@
   } on-error={}
 }
 
+:local credentialFile "evofw-credentials.json"
+:if ([:len [/file find name="flash"]] > 0) do={
+  :local flashType [/file get [find name="flash"] type]
+  :if ($flashType = "disk" || $flashType = "directory") do={ :set credentialFile "flash/evofw-credentials.json" }
+}
+:local credentials {"origin"=$EvofwCpUrl}
+:if ([:len [/file find name=$credentialFile]] > 0) do={
+  :set credentials [:deserialize [/file get [find name=$credentialFile] contents] from=json options=json.no-string-conversion]
+  :if (($credentials->"origin") != $EvofwCpUrl) do={ :error "credential controller mismatch" }
+  :if ([:typeof ($credentials->"active_token")] = "str" && [:len ($credentials->"active_token")] >= 16) do={ :set EvofwToken ($credentials->"active_token"); :set already 1 }
+}
 :if ($already = 0) do={
-  :local token ("evofw_" . [:rndstr length=32 from="0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"])
+  :if ([:typeof $EvofwInstallLinkId] != "str" || [:len $EvofwInstallLinkId] = 0) do={ :error "scoped install link required" }
-  :if ([:len $token] < 20) do={
+  :if ([:typeof ($credentials->"enrollment_token")] != "str") do={
-    :set token ("evofw_" . [:tostr [/system clock get time]] . [:tostr [/system resource get cpu-load]] . [:tostr [/system resource get free-memory]] . [:tostr [:rndnum from=100000 to=999999]])
+    :set ($credentials->"enrollment_token") ("evofw_" . [:rndstr length=43 from="0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"])
-  }
+    :set ($credentials->"install_link_id") $EvofwInstallLinkId
-
+    :if ([:len [/file find name=$credentialFile]] = 0) do={ /file add name=$credentialFile type=file }
-  :local body ("{\"name\":\"" . $EvofwName . "\",\"hostname\":\"" . [/system identity get name] . "\",\"platform\":\"mikrotik\",\"token\":\"" . $token . "\",\"client_version\":\"rsc/6\"")
+    /file set [find name=$credentialFile] contents=[:serialize $credentials to=json options=json.no-string-conversion]
-  :if ([:typeof $EvofwInstallLinkId] != "nothing" && [:len $EvofwInstallLinkId] > 0) do={
-    :set body ($body . ",\"install_link_id\":\"" . $EvofwInstallLinkId . "\"")
-  }
-  :set body ($body . "}")
-
-  :do {
-    /tool fetch url=($EvofwCpUrl . "/v1/agent/enroll") http-method=post http-header-field=("Content-Type: application/json,X-EvoFW-Seed: " . $EvofwSeed) http-data=$body keep-result=no
-  } on-error={
-    :error "evofw: enroll failed — check EvofwCpUrl / EvofwSeed / connectivity"
   }
+  :if (($credentials->"install_link_id") != $EvofwInstallLinkId) do={ :error "pending enrollment belongs to another link" }
+  :local token ($credentials->"enrollment_token")
+  :local body {"name"=$EvofwName;"hostname"=[/system identity get name];"platform"="mikrotik";"token"=$token;"client_version"="rsc/10";"install_link_id"=$EvofwInstallLinkId}
+  :local result [/tool fetch url=($EvofwCpUrl . "/v1/agent/enroll") http-method=post http-header-field=("Content-Type: application/json,X-EvoFW-Enrollment-Credential: " . $EvofwEnrollmentCredential) http-data=[:serialize $body to=json options=json.no-string-conversion] output=user as-value duration=1m]
+  :if (($result->"status") != "finished") do={ :error "enrollment ACK unavailable; saved token retained" }
+  :local ack [:deserialize ($result->"data") from=json options=json.no-string-conversion]
+  :if ([:typeof ($ack->"client_id")] != "str" || [:len ($ack->"client_id")] = 0) do={ :error "invalid enrollment ACK" }
+  :set ($credentials->"active_token") $token
+  :set ($credentials->"client_id") ($ack->"client_id")
+  :set ($credentials->"enrollment_token") ""
+  /file set [find name=$credentialFile] contents=[:serialize $credentials to=json options=json.no-string-conversion]
   :set EvofwToken $token
 } else={
-  :put "evofw: already enrolled — updating single sync script (token kept)"
+  :put "evofw: already enrolled - updating single sync script (token kept)"
 }
 
 :set EvofwLastHash ""
 
 # Filter rules (idempotent by comment) — always at the top of each chain.
 # Deny path: hit (add-src → EVOFW_HITS) then drop. Allow/default unchanged.
-:do { /ip firewall filter remove [find comment~"^evofw-"] } on-error={}
+# Preserve active owned rules and their counters when refreshing the installer.
 
-:local inputFirst [/ip firewall filter find where chain=input]
+:local inputFirst [/ip firewall filter find where chain=input and dynamic=no]
 :local inputPb ""
 :if ([:len $inputFirst] > 0) do={ :set inputPb ($inputFirst->0) }
 
-:local forwardFirst [/ip firewall filter find where chain=forward]
+:local forwardFirst [/ip firewall filter find where chain=forward and dynamic=no]
 :local forwardPb ""
 :if ([:len $forwardFirst] > 0) do={ :set forwardPb ($forwardFirst->0) }
 
 # place-before same anchor repeatedly → earlier adds stay above later ones.
 :if ([:len $inputPb] > 0) do={
-  /ip firewall filter add chain=input action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-input disabled=no place-before=$inputPb
+  :if ([:len [/ip firewall filter find comment=evofw-deny-hit-input]] = 0) do={ /ip firewall filter add chain=input action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-input disabled=no place-before=$inputPb }
-  /ip firewall filter add chain=input action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-input disabled=no place-before=$inputPb
+  :if ([:len [/ip firewall filter find comment=evofw-deny-drop-input]] = 0) do={ /ip firewall filter add chain=input action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-input disabled=no place-before=$inputPb }
 } else={
-  /ip firewall filter add chain=input action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-input disabled=no
+  :if ([:len [/ip firewall filter find comment=evofw-deny-hit-input]] = 0) do={ /ip firewall filter add chain=input action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-input disabled=no }
-  /ip firewall filter add chain=input action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-input disabled=no
+  :if ([:len [/ip firewall filter find comment=evofw-deny-drop-input]] = 0) do={ /ip firewall filter add chain=input action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-input disabled=no }
 }
 
 :if ([:len $forwardPb] > 0) do={
-  /ip firewall filter add chain=forward action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-forward disabled=no place-before=$forwardPb
+  :if ([:len [/ip firewall filter find comment=evofw-deny-hit-forward]] = 0) do={ /ip firewall filter add chain=forward action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-forward disabled=no place-before=$forwardPb }
-  /ip firewall filter add chain=forward action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-forward disabled=no place-before=$forwardPb
+  :if ([:len [/ip firewall filter find comment=evofw-deny-drop-forward]] = 0) do={ /ip firewall filter add chain=forward action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-forward disabled=no place-before=$forwardPb }
-  /ip firewall filter add chain=forward action=accept src-address-list=EVOFW_ALLOW comment=evofw-allow-accept-forward disabled=no place-before=$forwardPb
+  :if ([:len [/ip firewall filter find comment=evofw-allow-accept-forward]] = 0) do={ /ip firewall filter add chain=forward action=accept src-address-list=EVOFW_ALLOW comment=evofw-allow-accept-forward disabled=no place-before=$forwardPb }
-  /ip firewall filter add chain=forward action=drop comment=evofw-default-drop-forward disabled=yes place-before=$forwardPb
+  :if ([:len [/ip firewall filter find comment=evofw-default-drop-forward]] = 0) do={ /ip firewall filter add chain=forward action=drop comment=evofw-default-drop-forward disabled=yes place-before=$forwardPb }
 } else={
-  /ip firewall filter add chain=forward action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-forward disabled=no
+  :if ([:len [/ip firewall filter find comment=evofw-deny-hit-forward]] = 0) do={ /ip firewall filter add chain=forward action=add-src-to-address-list address-list=EVOFW_HITS address-list-timeout=1h src-address-list=EVOFW_DENY comment=evofw-deny-hit-forward disabled=no }
-  /ip firewall filter add chain=forward action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-forward disabled=no
+  :if ([:len [/ip firewall filter find comment=evofw-deny-drop-forward]] = 0) do={ /ip firewall filter add chain=forward action=drop src-address-list=EVOFW_DENY comment=evofw-deny-drop-forward disabled=no }
-  /ip firewall filter add chain=forward action=accept src-address-list=EVOFW_ALLOW comment=evofw-allow-accept-forward disabled=no
+  :if ([:len [/ip firewall filter find comment=evofw-allow-accept-forward]] = 0) do={ /ip firewall filter add chain=forward action=accept src-address-list=EVOFW_ALLOW comment=evofw-allow-accept-forward disabled=no }
-  /ip firewall filter add chain=forward action=drop comment=evofw-default-drop-forward disabled=yes
+  :if ([:len [/ip firewall filter find comment=evofw-default-drop-forward]] = 0) do={ /ip firewall filter add chain=forward action=drop comment=evofw-default-drop-forward disabled=yes }
 }
 
 # Build sync body as a template script, then prepend baked credentials → one script
@@ -106,105 +119,495 @@
   :global EvofwCpUrl
   :global EvofwToken
   :global EvofwLastHash
+  :local credentialFile "evofw-credentials.json"
+  :if ([:len [/file find name="flash"]] > 0) do={
+    :local flashType [/file get [find name="flash"] type]
+    :if ($flashType = "disk" || $flashType = "directory") do={ :set credentialFile "flash/evofw-credentials.json" }
+  }
+  :local credentials {"origin"=$EvofwCpUrl;"active_token"=$EvofwToken}
+  :if ([:len [/file find name=$credentialFile]] > 0) do={
+    :set credentials [:deserialize [/file get [find name=$credentialFile] contents] from=json options=json.no-string-conversion]
+    :if (($credentials->"origin") != $EvofwCpUrl || [:typeof ($credentials->"active_token")] != "str") do={ :error "invalid active credential state" }
+    :set EvofwToken ($credentials->"active_token")
+  }
   :if ([:typeof $EvofwCpUrl] = "nothing" || [:len $EvofwCpUrl] = 0 || [:typeof $EvofwToken] = "nothing" || [:len $EvofwToken] = 0) do={
     :log error "evofw: missing EvofwCpUrl/EvofwToken"
     :error "evofw missing credentials"
   }
   :if ([:typeof $EvofwLastHash] = "nothing") do={ :set EvofwLastHash "" }
 
+  :if ([/system script job print count-only where script="evofw-sync"] > 1) do={ :error "evofw sync already running" }
+  # Resume the saved exchange before policy or telemetry. Failed ACK keeps active token unchanged.
+  :do {
+    :if ([:typeof ($credentials->"rotation")] != "array") do={
+      :local result [/tool fetch url=($EvofwCpUrl . "/v1/agent/credentials") http-header-field=("Authorization: Bearer " . $EvofwToken) output=user as-value duration=1m]
+      :if (($result->"status") = "finished") do={
+        :local status [:deserialize ($result->"data") from=json options=json.no-string-conversion]
+        :if ([:typeof ($status->"rotation")] = "array") do={
+          :set ($credentials->"client_id") ($status->"client_id")
+          :set ($credentials->"rotation") {"request_id"=(($status->"rotation")->"request_id");"old_token"=$EvofwToken;"next_token"=("evofw_" . [:rndstr length=43 from="0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"])}
+          :if ([:len [/file find name=$credentialFile]] = 0) do={ /file add name=$credentialFile type=file }
+          /file set [find name=$credentialFile] contents=[:serialize $credentials to=json options=json.no-string-conversion]
+        }
+      }
+    }
+  } on-error={ :if ([:typeof ($credentials->"rotation")] = "array") do={ :error "cannot persist pending credential exchange" } }
+  :if ([:typeof ($credentials->"rotation")] = "array") do={
+    :local rotation ($credentials->"rotation")
+    :local body {"request_id"=($rotation->"request_id");"next_token"=($rotation->"next_token")}
+    :local ack [:toarray ""]
+    :local abandoned false
+    :do {
+      :local result [/tool fetch url=($EvofwCpUrl . "/v1/agent/token-rotation") http-method=post http-header-field=("Content-Type: application/json,Authorization: Bearer " . ($rotation->"old_token")) http-data=[:serialize $body to=json options=json.no-string-conversion] output=user as-value duration=1m]
+      :if (($result->"status") != "finished") do={ :error "credential exchange pending" }
+      :set ack [:deserialize ($result->"data") from=json options=json.no-string-conversion]
+    } on-error={
+      :do {
+        # After replay expiry, the saved candidate can prove that it is already active.
+        :local proof [/tool fetch url=($EvofwCpUrl . "/v1/agent/credentials") http-header-field=("Authorization: Bearer " . ($rotation->"next_token")) output=user as-value duration=1m]
+        :if (($proof->"status") != "finished") do={ :error "credential recovery pending" }
+        :local identity [:deserialize ($proof->"data") from=json options=json.no-string-conversion]
+        :if (($identity->"client_id") != ($credentials->"client_id") || (($identity->"completed_rotation")->"request_id") != ($rotation->"request_id")) do={ :error "candidate identity mismatch" }
+        :set ack {"ok"=true;"request_id"=($rotation->"request_id");"client_id"=($credentials->"client_id")}
+      } on-error={
+        # An authenticated old-token read proves no exchange committed. Never
+        # discard a request still active on the controller after a transient failure.
+        :local proof [/tool fetch url=($EvofwCpUrl . "/v1/agent/credentials") http-header-field=("Authorization: Bearer " . ($rotation->"old_token")) output=user as-value duration=1m]
+        :if (($proof->"status") != "finished") do={ :error "old credential recovery pending" }
+        :local identity [:deserialize ($proof->"data") from=json options=json.no-string-conversion]
+        :if (($identity->"client_id") != ($credentials->"client_id")) do={ :error "old credential identity mismatch" }
+        :if ([:typeof ($identity->"rotation")] = "array" && (($identity->"rotation")->"request_id") = ($rotation->"request_id")) do={ :error "exchange request still active" }
+        :set ($credentials->"rotation") ""
+        /file set [find name=$credentialFile] contents=[:serialize $credentials to=json options=json.no-string-conversion]
+        :set abandoned true
+        # Continue policy/telemetry using the proven old credential; poll newer request next sync.
+      }
+    }
+    :if (!$abandoned) do={
+    :if (($ack->"ok") != true || ($ack->"request_id") != ($rotation->"request_id") || ($ack->"client_id") != ($credentials->"client_id")) do={ :error "credential ACK mismatch" }
+    :set ($credentials->"active_token") ($rotation->"next_token")
+    :set ($credentials->"rotation") ""
+    /file set [find name=$credentialFile] contents=[:serialize $credentials to=json options=json.no-string-conversion]
+    :set EvofwToken ($credentials->"active_token")
+    }
+  }
+  :global EvofwBootId
+  :global EvofwTrafficEpoch
+  :global EvofwLastObservedUptime
+  :global EvofwRuleCounters
+  :if ([:typeof $EvofwRuleCounters] != "array") do={ :set EvofwRuleCounters [:toarray ""] }
+  :global EvofwAppliedRevision
+  :if ([:typeof $EvofwBootId] = "nothing") do={ :set EvofwBootId [:rndstr length=32]; :set EvofwTrafficEpoch [:rndstr length=32] }
+  :local stateFile "evofw-telemetry.json"
+  :if ([:len [/file find name="flash"]] > 0) do={
+    :local flashType [/file get [find name="flash"] type]
+    :if ($flashType = "disk" || $flashType = "directory") do={ :set stateFile "flash/evofw-telemetry.json" }
+  }
+  :local telemetryState {"next_sequence"=0;"pending"=""}
+  :if ([:len [/file find name=$stateFile]] > 0) do={
+    # Corrupt state stops sync; never reset a persisted sequence silently.
+    :set telemetryState [:deserialize [/file get [find name=$stateFile] contents] from=json options=json.no-string-conversion]
+  }
+  :local sendPending do={
+    :global EvofwCpUrl
+    :global EvofwToken
+    :global EvofwSequenceFloor
+    :set EvofwSequenceFloor -1
+    :local state $state
+    :local body ($state->"pending")
+    :if ([:len $body] = 0) do={ :return true }
+    :local expected [:deserialize $body from=json options=json.no-string-conversion]
+    :local delivered false
+    :do {
+      :local result [/tool fetch url=($EvofwCpUrl . "/v1/agent/apply-report") http-method=post http-header-field=("Authorization: Bearer " . $EvofwToken . ",Content-Type: application/json") http-data=$body output=user as-value duration=1m]
+      :if (($result->"status") = "finished") do={
+        :local ack [:deserialize ($result->"data") from=json options=json.no-string-conversion]
+        :if (($ack->"ok") = true && ($ack->"report_id") = ($expected->"report_id") && ($ack->"sequence") = ($expected->"sequence") && (($ack->"accepted") = true || ($ack->"duplicate") = true || ($ack->"reason") = "stale_sequence")) do={ :set delivered true }
+        :if ($delivered && ($ack->"reason") = "stale_sequence") do={
+          :local floor ($ack->"sequence_floor")
+          :if ([:typeof $floor] != "num" || $floor < ($expected->"sequence") || $floor >= 9007199254740991) do={ :return false }
+          :set EvofwSequenceFloor $floor
+        }
+      }
+    } on-error={ :set delivered false }
+    :return $delivered
+  }
+  :local resyncSequence do={
+    :global EvofwSequenceFloor
+    :global EvofwTrafficEpoch
+    :local state $state
+    :if ([:typeof $EvofwSequenceFloor] = "num" && $EvofwSequenceFloor >= ($state->"next_sequence")) do={
+      :set ($state->"next_sequence") ($EvofwSequenceFloor + 1)
+      :set ($state->"coverage_unknown") true
+      :set EvofwTrafficEpoch [:rndstr length=32]
+    }
+    :return $state
+  }
+  :if ([:len ($telemetryState->"pending")] > 0) do={
+    :if ([$sendPending state=$telemetryState]) do={
+      :set ($telemetryState->"pending") ""
+    :set telemetryState [$resyncSequence state=$telemetryState]
+      /file set [find name=$stateFile] contents=[:serialize $telemetryState to=json options=json.no-string-conversion]
+    } else={ :log warning "evofw: pending telemetry ACK not confirmed; retry next sync"; :return }
+  }
+  :if ([:typeof ($telemetryState->"next_sequence")] != "num" || ($telemetryState->"next_sequence") < 0 || ($telemetryState->"next_sequence") >= 9007199254740991) do={ :error "invalid/exhausted telemetry sequence" }
+  :global EvofwRenderedHash
+  :global EvofwRenderedRevision
+  :global EvofwRenderedIpv6
+  :global EvofwAppliedIpv6
+  :global EvofwApplyErrorCode
+  :set EvofwApplyErrorCode "apply_failed"
+  :local readCounters do={
+    :global EvofwAppliedIpv6
+    :global EvofwRuleCounters
+    :global EvofwTrafficEpoch
+    :local observed [:toarray ""]
+    :local dropped 0
+    :local accepted 0
+    :local valid true
+    :do {
+      :local id [/ip firewall filter find comment=evofw-deny-drop-input]
+      :if ([:len $id] != 1 || [/ip firewall filter get $id chain] != "input" || [/ip firewall filter get $id action] != "drop") do={ :error "counter rule unavailable" }
+      :local n [/ip firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ip:evofw-deny-drop-input") $n
+      :if ($dropped > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set dropped ($dropped + $n)
+      :local id [/ip firewall filter find comment=evofw-deny-drop-forward]
+      :if ([:len $id] != 1 || [/ip firewall filter get $id chain] != "forward" || [/ip firewall filter get $id action] != "drop") do={ :error "counter rule unavailable" }
+      :local n [/ip firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ip:evofw-deny-drop-forward") $n
+      :if ($dropped > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set dropped ($dropped + $n)
+      :local id [/ip firewall filter find comment=evofw-default-drop-forward]
+      :if ([:len $id] != 1 || [/ip firewall filter get $id chain] != "forward" || [/ip firewall filter get $id action] != "drop") do={ :error "counter rule unavailable" }
+      :local n [/ip firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ip:evofw-default-drop-forward") $n
+      :if ($dropped > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set dropped ($dropped + $n)
+      :local id [/ip firewall filter find comment=evofw-allow-accept-forward]
+      :if ([:len $id] != 1 || [/ip firewall filter get $id chain] != "forward" || [/ip firewall filter get $id action] != "accept") do={ :error "counter rule unavailable" }
+      :local n [/ip firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ip:evofw-allow-accept-forward") $n
+      :if ($accepted > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set accepted ($accepted + $n)
+    } on-error={ :set valid false }
+    :if ($EvofwAppliedIpv6 = true) do={
+    :do {
+      :local id [/ipv6 firewall filter find comment=evofw-deny-drop-input]
+      :if ([:len $id] != 1 || [/ipv6 firewall filter get $id chain] != "input" || [/ipv6 firewall filter get $id action] != "drop") do={ :error "counter rule unavailable" }
+      :local n [/ipv6 firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ipv6:evofw-deny-drop-input") $n
+      :if ($dropped > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set dropped ($dropped + $n)
+      :local id [/ipv6 firewall filter find comment=evofw-deny-drop-forward]
+      :if ([:len $id] != 1 || [/ipv6 firewall filter get $id chain] != "forward" || [/ipv6 firewall filter get $id action] != "drop") do={ :error "counter rule unavailable" }
+      :local n [/ipv6 firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ipv6:evofw-deny-drop-forward") $n
+      :if ($dropped > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set dropped ($dropped + $n)
+      :local id [/ipv6 firewall filter find comment=evofw-default-drop-forward]
+      :if ([:len $id] != 1 || [/ipv6 firewall filter get $id chain] != "forward" || [/ipv6 firewall filter get $id action] != "drop") do={ :error "counter rule unavailable" }
+      :local n [/ipv6 firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ipv6:evofw-default-drop-forward") $n
+      :if ($dropped > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set dropped ($dropped + $n)
+      :local id [/ipv6 firewall filter find comment=evofw-allow-accept-forward]
+      :if ([:len $id] != 1 || [/ipv6 firewall filter get $id chain] != "forward" || [/ipv6 firewall filter get $id action] != "accept") do={ :error "counter rule unavailable" }
+      :local n [/ipv6 firewall filter get $id packets]
+      :if ($n < 0 || $n > 9007199254740991) do={ :error "unsafe counter" }
+      :set ($observed->"ipv6:evofw-allow-accept-forward") $n
+      :if ($accepted > (9007199254740991 - $n)) do={ :error "unsafe total" }
+      :set accepted ($accepted + $n)
+    } on-error={ :set valid false }
+    }
+    :if (!$valid) do={ :return {"valid"=false} }
+    # Commit complete observations only: omissions must not erase reset baselines.
+    :local reset false
+    :foreach key,value in=$observed do={
+      :local previous ($EvofwRuleCounters->$key)
+      :if ([:typeof $previous] = "num" && $value < $previous) do={ :set reset true }
+    }
+    :if ($reset) do={ :set EvofwTrafficEpoch [:rndstr length=32] }
+    :set EvofwRuleCounters $observed
+    :return {"valid"=true;"reset"=$reset;"packets_dropped"=$dropped;"packets_accepted"=$accepted}
+  }
   :local syncOk 0
   :do {
-    :local result [/tool fetch url=($EvofwCpUrl . "/v1/agent/policy") http-header-field=("Authorization: Bearer " . $EvofwToken . ",Accept: application/json") output=user as-value]
+    :local jsonFile "evofw-policy-next.json"
-    :if (($result->"status") != "finished") do={
+    :local result [/tool fetch url=($EvofwCpUrl . "/v1/agent/policy?transport=routeros") http-header-field=("Authorization: Bearer " . $EvofwToken . ",Accept: application/json,Range: bytes=0-8388608") dst-path=$jsonFile as-value duration=1m]
-      :log warning ("evofw: policy fetch status=" . ($result->"status"))
+    :if (($result->"status") != "finished") do={ :error "policy fetch failed" }
-      :error "policy fetch failed"
+    :local fileSize [/file get [find name=$jsonFile] size]
+    :if ([:typeof $fileSize] != "num" || $fileSize < 2 || $fileSize > 8388608) do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "policy download budget exceeded" }
+    :local raw ""
+    :local offset 0
+    :while ($offset < $fileSize) do={
+      :local chunk ([/file read file=$jsonFile offset=$offset chunk-size=32768 as-value]->"data")
+      :if ([:len $chunk] = 0) do={ :error "policy file truncated" }
+      :set raw ($raw . $chunk)
+      :set offset ($offset + [:len $chunk])
+      :if ($offset > 8388608) do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "policy byte budget exceeded" }
+    }
+    :local p [:deserialize $raw from=json options=json.no-string-conversion]
+    :if ([:typeof ($p->"error")] = "array") do={
+      :if ((($p->"error")->"code") = "POLICY_BUDGET_EXCEEDED") do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "controller policy budget exceeded" }
+      :error "unexpected policy error envelope"
     }
-    :local raw ($result->"data")
-    :if ([:len $raw] < 10) do={ :error "empty policy body" }
-    :local p [:deserialize $raw from=json]
     :local hash ($p->"hash")
-    :if ([:typeof $hash] = "nothing" || [:len $hash] = 0) do={ :error "policy missing hash (pending?)" }
+    :if ([:typeof $hash] != "str" || [:len $hash] != 71 || !($hash~"^sha256:[0-9a-f]+$")) do={ :error "invalid policy hash" }
-
-    :if ($hash = $EvofwLastHash) do={
-      :log info ("evofw: unchanged " . $hash)
-      :set syncOk 1
-    } else={
     :local deny ($p->"deny_cidrs")
     :local allow ($p->"allow_cidrs")
-      :do { /ip firewall address-list remove [find list=EVOFW_DENY] } on-error={}
+    :if ([:typeof $deny] != "array" || [:typeof $allow] != "array") do={ :error "invalid policy network arrays" }
-      :do { /ip firewall address-list remove [find list=EVOFW_ALLOW] } on-error={}
-      :foreach a in=$deny do={
-        :do { /ip firewall address-list add list=EVOFW_DENY address=$a comment=evofw } on-error={}
-      }
-      :foreach a in=$allow do={
-        :do { /ip firewall address-list add list=EVOFW_ALLOW address=$a comment=evofw } on-error={}
-      }
     :local da ($p->"default_action")
-      :do { /ip firewall filter set [find comment=evofw-deny-hit-input] disabled=no } on-error={}
+    :if ($da != "accept" && $da != "drop") do={ :error "invalid default action" }
-      :do { /ip firewall filter set [find comment=evofw-deny-drop-input] disabled=no } on-error={}
+    :local rawCount ([:len $deny] + [:len $allow])
-      :do { /ip firewall filter set [find comment=evofw-deny-hit-forward] disabled=no } on-error={}
+    :local portRules ($p->"port_rules")
-      :do { /ip firewall filter set [find comment=evofw-deny-drop-forward] disabled=no } on-error={}
+    :if ([:typeof $portRules] != "array") do={ :error "invalid policy port rules" }
-      :do { /ip firewall filter set [find comment=evofw-allow-accept-forward] disabled=no } on-error={}
+    :if ([:len $portRules] > 1000) do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "port rule budget exceeded" }
-      :if ($da = "drop") do={
+    :foreach rule in=$portRules do={ :set rawCount ($rawCount + [:len ($rule->"src_cidrs")]) }
-        :do { /ip firewall filter set [find comment=evofw-default-drop-forward] disabled=no } on-error={}
+    :if ($rawCount > 200000) do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "raw network membership budget exceeded" }
-      } else={
+    :local countDistinct do={
-        :do { /ip firewall filter set [find comment=evofw-default-drop-forward] disabled=yes } on-error={}
+      :local seen [:toarray ""]
+      :local count 0
+      :foreach value in=$values do={
+        :if ([:typeof $value] != "str" || [:len $value] > 128) do={ :error "invalid network text" }
+        :if ([:typeof ($seen->$value)] != "bool") do={ :set ($seen->$value) true; :set count ($count + 1) }
       }
-      :set EvofwLastHash $hash
+      :return $count
-      :log info ("evofw: applied " . $hash)
+    }
+    :local effectiveCount ([$countDistinct values=$deny] + [$countDistinct values=$allow])
+    :foreach rule in=$portRules do={ :set effectiveCount ($effectiveCount + [$countDistinct values=($rule->"src_cidrs")]) }
+    :if ($effectiveCount > 50000) do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "effective network membership budget exceeded" }
+    :if ([:len ($p->"port_rules")] > 0) do={ :set EvofwApplyErrorCode "UNSUPPORTED_CAPABILITY"; :error "Port ACL is unsupported on MikroTik" }
+    :local v6Enabled false
+    :do { :set v6Enabled (![/ipv6 settings get disable-ipv6]) } on-error={ :set v6Enabled false }
+    :local needs6 false
+    :foreach a in=$deny do={ :if ([:typeof [:find $a ":"]] != "nil") do={ :set needs6 true } }
+    :foreach a in=$allow do={ :if ([:typeof [:find $a ":"]] != "nil") do={ :set needs6 true } }
+    :if ($needs6 && !$v6Enabled) do={ :set EvofwApplyErrorCode "UNSUPPORTED_CAPABILITY"; :error "IPv6 enforcement unavailable" }
+    :local drift 0
+    :do {
+      :local rule [/ip firewall filter find comment=evofw-deny-drop-input]
+      :if ([:len $rule] != 1 || [/ip firewall filter get $rule chain] != "input" || [/ip firewall filter get $rule action] != "drop" || [/ip firewall filter get $rule disabled] != false) do={ :error "owned filter chain/action drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ip firewall filter print as-value where comment=[/ip firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local rule [/ip firewall filter find comment=evofw-deny-drop-forward]
+      :if ([:len $rule] != 1 || [/ip firewall filter get $rule chain] != "forward" || [/ip firewall filter get $rule action] != "drop" || [/ip firewall filter get $rule disabled] != false) do={ :error "owned filter chain/action drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ip firewall filter print as-value where comment=[/ip firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local rule [/ip firewall filter find comment=evofw-allow-accept-forward]
+      :if ([:len $rule] != 1 || [/ip firewall filter get $rule chain] != "forward" || [/ip firewall filter get $rule action] != "accept" || [/ip firewall filter get $rule disabled] != false) do={ :error "owned filter chain/action drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ip firewall filter print as-value where comment=[/ip firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local rule [/ip firewall filter find comment=evofw-default-drop-forward]
+      :if ([:len $rule] != 1 || [/ip firewall filter get $rule chain] != "forward" || [/ip firewall filter get $rule action] != "drop" || [/ip firewall filter get $rule disabled] != ($da = "accept")) do={ :error "owned filter chain/action drift" }
+      :if ([/ip firewall filter get $rule src-address-list] != "") do={ :error "owned default matcher drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ip firewall filter print as-value where comment=[/ip firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local ordered [/ip firewall filter find where chain=input and dynamic=no]
+      :if (($ordered->0) != ([/ip firewall filter find comment=evofw-deny-drop-input]->0)) do={ :error "owned filter order drift" }
+      :local ordered [/ip firewall filter find where chain=forward and dynamic=no]
+      :if (($ordered->0) != ([/ip firewall filter find comment=evofw-deny-drop-forward]->0)) do={ :error "owned filter order drift" }
+      :if (($ordered->1) != ([/ip firewall filter find comment=evofw-allow-accept-forward]->0)) do={ :error "owned filter order drift" }
+      :if (($ordered->2) != ([/ip firewall filter find comment=evofw-default-drop-forward]->0)) do={ :error "owned filter order drift" }
+      :local denyList [/ip firewall filter get [find comment=evofw-deny-drop-input] src-address-list]
+      :local forwardList [/ip firewall filter get [find comment=evofw-deny-drop-forward] src-address-list]
+      :local allowList [/ip firewall filter get [find comment=evofw-allow-accept-forward] src-address-list]
+      :if ($denyList != $forwardList || !($denyList~"^EVOFW_DENY_") || !($allowList~"^EVOFW_ALLOW_")) do={ :error "owned source bank drift" }
+      :local expectedCount 0
+      :foreach a in=$deny do={ :if ([:typeof [:find $a ":"]] = "nil") do={
+        :set expectedCount ($expectedCount + 1)
+        :if ([:len [/ip firewall address-list find list=$denyList address=$a]] != 1) do={ :error "owned list membership drift" }
+      } }
+      :if ([:len [/ip firewall address-list find list=$denyList]] != $expectedCount) do={ :error "owned list size drift" }
+      :local expectedCount 0
+      :foreach a in=$allow do={ :if ([:typeof [:find $a ":"]] = "nil") do={
+        :set expectedCount ($expectedCount + 1)
+        :if ([:len [/ip firewall address-list find list=$allowList address=$a]] != 1) do={ :error "owned list membership drift" }
+      } }
+      :if ([:len [/ip firewall address-list find list=$allowList]] != $expectedCount) do={ :error "owned list size drift" }
+    } on-error={ :set drift 1 }
+    :if ($v6Enabled) do={
+    :do {
+      :local rule [/ipv6 firewall filter find comment=evofw-deny-drop-input]
+      :if ([:len $rule] != 1 || [/ipv6 firewall filter get $rule chain] != "input" || [/ipv6 firewall filter get $rule action] != "drop" || [/ipv6 firewall filter get $rule disabled] != false) do={ :error "owned filter chain/action drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ipv6 firewall filter print as-value where comment=[/ipv6 firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local rule [/ipv6 firewall filter find comment=evofw-deny-drop-forward]
+      :if ([:len $rule] != 1 || [/ipv6 firewall filter get $rule chain] != "forward" || [/ipv6 firewall filter get $rule action] != "drop" || [/ipv6 firewall filter get $rule disabled] != false) do={ :error "owned filter chain/action drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ipv6 firewall filter print as-value where comment=[/ipv6 firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local rule [/ipv6 firewall filter find comment=evofw-allow-accept-forward]
+      :if ([:len $rule] != 1 || [/ipv6 firewall filter get $rule chain] != "forward" || [/ipv6 firewall filter get $rule action] != "accept" || [/ipv6 firewall filter get $rule disabled] != false) do={ :error "owned filter chain/action drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ipv6 firewall filter print as-value where comment=[/ipv6 firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local rule [/ipv6 firewall filter find comment=evofw-default-drop-forward]
+      :if ([:len $rule] != 1 || [/ipv6 firewall filter get $rule chain] != "forward" || [/ipv6 firewall filter get $rule action] != "drop" || [/ipv6 firewall filter get $rule disabled] != ($da = "accept")) do={ :error "owned filter chain/action drift" }
+      :if ([/ipv6 firewall filter get $rule src-address-list] != "") do={ :error "owned default matcher drift" }
+      :local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}
+      :foreach row in=[/ipv6 firewall filter print as-value where comment=[/ipv6 firewall filter get $rule comment]] do={
+        :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }
+        :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "owned matcher drift" } }
+      }
+      :local ordered [/ipv6 firewall filter find where chain=input and dynamic=no]
+      :if (($ordered->0) != ([/ipv6 firewall filter find comment=evofw-deny-drop-input]->0)) do={ :error "owned filter order drift" }
+      :local ordered [/ipv6 firewall filter find where chain=forward and dynamic=no]
+      :if (($ordered->0) != ([/ipv6 firewall filter find comment=evofw-deny-drop-forward]->0)) do={ :error "owned filter order drift" }
+      :if (($ordered->1) != ([/ipv6 firewall filter find comment=evofw-allow-accept-forward]->0)) do={ :error "owned filter order drift" }
+      :if (($ordered->2) != ([/ipv6 firewall filter find comment=evofw-default-drop-forward]->0)) do={ :error "owned filter order drift" }
+      :local denyList [/ipv6 firewall filter get [find comment=evofw-deny-drop-input] src-address-list]
+      :local forwardList [/ipv6 firewall filter get [find comment=evofw-deny-drop-forward] src-address-list]
+      :local allowList [/ipv6 firewall filter get [find comment=evofw-allow-accept-forward] src-address-list]
+      :if ($denyList != $forwardList || !($denyList~"^EVOFW_DENY_") || !($allowList~"^EVOFW_ALLOW_")) do={ :error "owned source bank drift" }
+      :local expectedCount 0
+      :foreach a in=$deny do={ :if ([:typeof [:find $a ":"]] != "nil") do={
+        :set expectedCount ($expectedCount + 1)
+        :if ([:len [/ipv6 firewall address-list find list=$denyList address=$a]] != 1) do={ :error "owned list membership drift" }
+      } }
+      :if ([:len [/ipv6 firewall address-list find list=$denyList]] != $expectedCount) do={ :error "owned list size drift" }
+      :local expectedCount 0
+      :foreach a in=$allow do={ :if ([:typeof [:find $a ":"]] != "nil") do={
+        :set expectedCount ($expectedCount + 1)
+        :if ([:len [/ipv6 firewall address-list find list=$allowList address=$a]] != 1) do={ :error "owned list membership drift" }
+      } }
+      :if ([:len [/ipv6 firewall address-list find list=$allowList]] != $expectedCount) do={ :error "owned list size drift" }
+    } on-error={ :set drift 1 }
+    }
+    :if ($hash = $EvofwLastHash && $drift = 0) do={
+      :set EvofwAppliedRevision ($p->"generation")
+      :set EvofwAppliedIpv6 $v6Enabled
+      :log info ("evofw: unchanged; both-family kernel verified " . $hash)
+      :set syncOk 1
+    } else={
+      # ACK a durable final snapshot before the compiler resets rule counters.
+      :if ([:len $EvofwLastHash] = 71) do={
+        :local finalCounters [$readCounters]
+        :local finalSeq ($telemetryState->"next_sequence")
+        :local finalUptime [:tonsec [/system resource get uptime]]
+        :local finalInterval 60000
+        :if ([:typeof $EvofwLastObservedUptime] != "nothing" && $finalUptime > $EvofwLastObservedUptime) do={ :set finalInterval (($finalUptime - $EvofwLastObservedUptime) / 1000000) }
+        :if ($finalInterval < 1) do={ :set finalInterval 1 }
+        :local observed ([/system clock get date] . "T" . [/system clock get time] . [/system clock get gmt-offset])
+        :local final {"telemetry_version"=2;"report_id"=[:rndstr length=32];"sequence"=$finalSeq;"boot_id"=$EvofwBootId;"observed_at"=$observed;"interval_ms"=$finalInterval;"counter_epoch"={"traffic"=$EvofwTrafficEpoch};"agent_version"="rsc/10";"source"="mikrotik";"status"="ok";"kernel_method"="address-list";"applied_hash"=$EvofwLastHash;"capabilities"={"telemetry-v2"};"quality"={"partial"=true;"truncated"=false;"unsupported"={"final_snapshot_before_apply";"file_commit_not_powerloss_atomic"};"lost_reports"=0}}
+        :if (($finalCounters->"reset") = true) do={ :set (($final->"quality")->"unsupported") {"final_snapshot_before_apply";"file_commit_not_powerloss_atomic";"counter_reset_coverage_unknown"} }
+        :if ($drift = 1) do={ :set ($final->"status") "error"; :set ($final->"apply_error") {"code"="KERNEL_DRIFT";"message"="Owned rule drift detected; repair pending";"stage"="verify";"retryable"=true} }
+        :if (($finalCounters->"valid") = true) do={ :set ($final->"packets_dropped") ($finalCounters->"packets_dropped"); :set ($final->"packets_accepted") ($finalCounters->"packets_accepted") }
+        :if ([:typeof $EvofwAppliedRevision] = "num") do={ :set ($final->"applied_revision") $EvofwAppliedRevision }
+        :set ($telemetryState->"next_sequence") ($finalSeq + 1)
+        :set ($telemetryState->"pending") [:serialize $final to=json options=json.no-string-conversion]
+        :if ([:len [/file find name=$stateFile]] = 0) do={ /file add name=$stateFile type=file }
+        /file set [find name=$stateFile] contents=[:serialize $telemetryState to=json options=json.no-string-conversion]
+        :set EvofwLastObservedUptime $finalUptime
+        :if ([$sendPending state=$telemetryState]) do={
+          :set ($telemetryState->"pending") ""
+          :set telemetryState [$resyncSequence state=$telemetryState]
+          /file set [find name=$stateFile] contents=[:serialize $telemetryState to=json options=json.no-string-conversion]
+        } else={ :error "final telemetry ACK pending; defer counter-resetting apply" }
+      }
+      :local file "evofw-policy-next.rsc"
+      :set EvofwRenderedHash ""
+      :set EvofwRenderedRevision -1
+      :local compiled [/tool fetch url=($EvofwCpUrl . "/v1/agent/policy.rsc") http-header-field=("Authorization: Bearer " . $EvofwToken) dst-path=$file as-value duration=1m]
+      :if (($compiled->"status") != "finished") do={ :error "compiled policy download failed" }
+      :local compiledSize [/file get [find name=$file] size]
+      :if ([:typeof $compiledSize] != "num" || $compiledSize > 8388608) do={ :set EvofwApplyErrorCode "POLICY_BUDGET_EXCEEDED"; :error "compiled policy byte budget exceeded" }
+      /import file-name=$file
+      # The second endpoint can render a newer policy. ACK its actual metadata,
+      # not the earlier JSON generation; blank metadata means import failed.
+      :if ([:len $EvofwRenderedHash] != 71 || [:typeof $EvofwRenderedRevision] != "num" || $EvofwRenderedRevision < 0) do={ :error "compiled policy did not complete" }
+      :set EvofwLastHash $EvofwRenderedHash
+      :set EvofwAppliedRevision $EvofwRenderedRevision
+      :set EvofwAppliedIpv6 $EvofwRenderedIpv6
+      :set EvofwTrafficEpoch [:rndstr length=32]
+      :set EvofwRuleCounters [:toarray ""]
       :set syncOk 1
     }
   } on-error={
-    :log warning "evofw: policy sync failed (pending approval, network, or JSON)"
+    :log warning "evofw: policy sync failed; last successful hash retained"
   }
 
+  :if ([:len ($telemetryState->"pending")] > 0) do={ :return }
+
+  # One immutable pending report is saved before sending. Address-list presence
+  # is never presented as measured IP packets. No separate post-ACK heartbeat.
+  :local status "error"
+  :if ($syncOk = 1) do={ :set status "ok" }
+  :local seq ($telemetryState->"next_sequence")
+  :local uptime [:tonsec [/system resource get uptime]]
+  :local interval 60000
+  :if ([:typeof $EvofwLastObservedUptime] != "nothing" && $uptime > $EvofwLastObservedUptime) do={ :set interval (($uptime - $EvofwLastObservedUptime) / 1000000) }
+  :if ($interval < 1) do={ :set interval 1 }
+  :local observed ([/system clock get date] . "T" . [/system clock get time] . [/system clock get gmt-offset])
+  :local unsupported {"ip_counters_presence_only";"port_counters_unavailable";"host_firewall_unavailable";"file_commit_not_powerloss_atomic"}
+  :local report {"telemetry_version"=2;"report_id"=[:rndstr length=32];"sequence"=$seq;"boot_id"=$EvofwBootId;"observed_at"=$observed;"interval_ms"=$interval;"agent_version"="rsc/10";"status"=$status;"kernel_method"="address-list";"source"="mikrotik"}
+  :local capabilities {"telemetry-v2";"policy-ipv4";"mikrotik-staged-apply"}
   :if ($syncOk = 1) do={
-    :local denyCnt [:len [/ip firewall address-list find list=EVOFW_DENY]]
+    :local denyCnt [:len [/ip firewall address-list find list=[/ip firewall filter get [find comment=evofw-deny-drop-input] src-address-list]]]
-    :local allowCnt [:len [/ip firewall address-list find list=EVOFW_ALLOW]]
+    :local allowCnt [:len [/ip firewall address-list find list=[/ip firewall filter get [find comment=evofw-allow-accept-forward] src-address-list]]]
-    :local cnt ($denyCnt + $allowCnt)
+    :set ($report->"prefix_count") ($denyCnt + $allowCnt)
-    # Cumulative counters from permanent drop/accept rules (not hit passthrough).
+    :local counters [$readCounters]
-    :local dropped 0
+    :if (($counters->"valid") = true) do={
-    :local accepted 0
+      :local dropped ($counters->"packets_dropped")
-    :do {
+      :local accepted ($counters->"packets_accepted")
-      :foreach i in=[/ip firewall filter find where comment~"^evofw-deny-drop-"] do={
+      :set ($report->"packets_dropped") $dropped
-        :set dropped ($dropped + [/ip firewall filter get $i packets])
+      :set ($report->"packets_accepted") $accepted
+      :set capabilities ($capabilities,{"traffic-counters"})
+      :if (($counters->"reset") = true) do={ :set unsupported ($unsupported,{"counter_reset_coverage_unknown"}) }
+    } else={ :set unsupported ($unsupported,{"traffic_counters_unavailable"}) }
+    :if ($EvofwAppliedIpv6 = true) do={
+      :set capabilities ($capabilities,{"policy-ipv6";"traffic-counters-ipv6"})
+      :local deny6 [:len [/ipv6 firewall address-list find list=[/ipv6 firewall filter get [find comment=evofw-deny-drop-input] src-address-list]]]
+      :local allow6 [:len [/ipv6 firewall address-list find list=[/ipv6 firewall filter get [find comment=evofw-allow-accept-forward] src-address-list]]]
+      :set ($report->"prefix_count") (($report->"prefix_count") + $deny6 + $allow6)
     }
-    } on-error={}
+    :if ([:len $EvofwLastHash] = 71) do={ :set ($report->"applied_hash") $EvofwLastHash }
-    :do {
+    :if ([:typeof $EvofwAppliedRevision] = "num") do={ :set ($report->"applied_revision") $EvofwAppliedRevision }
-      :foreach i in=[/ip firewall filter find where comment=evofw-default-drop-forward] do={
+  } else={
-        :set dropped ($dropped + [/ip firewall filter get $i packets])
+    :set ($report->"apply_error") {"code"=$EvofwApplyErrorCode;"message"="Policy validation, capability, import or apply failed; previous hash retained";"stage"="routeros";"retryable"=true}
-      }
-    } on-error={}
-    :do {
-      :foreach i in=[/ip firewall filter find where comment=evofw-allow-accept-forward] do={
-        :set accepted ($accepted + [/ip firewall filter get $i packets])
-      }
-    } on-error={}
-    # EVOFW_HITS: real src /32 that matched deny (timeout 1h). Do not wipe on policy rebuild.
-    :local hits "["
-    :local hitN 0
-    :do {
-      :foreach i in=[/ip firewall address-list find list=EVOFW_HITS] do={
-        :if ($hitN < 200) do={
-          :local a [/ip firewall address-list get $i address]
-          :if ([:len $a] > 0) do={
-            :if ($hitN > 0) do={ :set hits ($hits . ",") }
-            :set hits ($hits . "{\"ip\":\"" . $a . "\",\"packets\":1}")
-            :set hitN ($hitN + 1)
-          }
-        }
-      }
-    } on-error={}
-    :set hits ($hits . "]")
-    :local report ("{\"status\":\"ok\",\"prefix_count\":" . $cnt . ",\"packets_dropped\":" . $dropped . ",\"packets_accepted\":" . $accepted . ",\"kernel_method\":\"address-list\",\"source\":\"mikrotik\",\"ip_hits\":" . $hits . "}")
-    :do {
-      /tool fetch url=($EvofwCpUrl . "/v1/agent/apply-report") http-method=post http-header-field=("Authorization: Bearer " . $EvofwToken . ",Content-Type: application/json") http-data=$report keep-result=no
-    } on-error={}
-    :do {
-      /tool fetch url=($EvofwCpUrl . "/v1/agent/heartbeat") http-method=post http-header-field=("Authorization: Bearer " . $EvofwToken . ",Content-Type: application/json") http-data="{\"source\":\"mikrotik\"}" keep-result=no
-    } on-error={}
-    :log info ("evofw: sync done deny=" . $denyCnt . " allow=" . $allowCnt . " hits=" . $hitN . " dropPkts=" . $dropped . " acceptPkts=" . $accepted)
   }
+  :set ($report->"counter_epoch") {"traffic"=$EvofwTrafficEpoch}
+  :set ($report->"capabilities") $capabilities
+  :if (($telemetryState->"coverage_unknown") = true) do={ :set unsupported ($unsupported,{"sequence_rollback_coverage_unknown"}) }
+  :set ($report->"quality") {"partial"=true;"truncated"=false;"unsupported"=$unsupported;"lost_reports"=0}
+  :set ($telemetryState->"next_sequence") ($seq + 1)
+  :set ($telemetryState->"coverage_unknown") false
+  :set ($telemetryState->"pending") [:serialize $report to=json options=json.no-string-conversion]
+  :if ([:len [/file find name=$stateFile]] = 0) do={ /file add name=$stateFile type=file }
+  /file set [find name=$stateFile] contents=[:serialize $telemetryState to=json options=json.no-string-conversion]
+  :set EvofwLastObservedUptime $uptime
+  :if ([$sendPending state=$telemetryState]) do={
+    :set ($telemetryState->"pending") ""
+    :set telemetryState [$resyncSequence state=$telemetryState]
+    /file set [find name=$stateFile] contents=[:serialize $telemetryState to=json options=json.no-string-conversion]
+  } else={ :log warning "evofw: saved pending telemetry; retry next sync" }
+
 }
 
 :local bodySrc [/system script get [find name="evofw-sync-body"] source]

apps/api/src/app.ts

@@ -9,6 +9,7 @@ import { AsyncTask, CronJob } from 'toad-scheduler'
 import type { AppConfig } from './config.js'
 import { loadConfig } from './config.js'
 import authPlugin from './plugins/auth.js'
+import operationalPlugin from './plugins/operational.js'
 import corsPlugin from './plugins/cors.js'
 import dbPlugin from './plugins/db.js'
 import errorHandlerPlugin from './plugins/error-handler.js'
@@ -16,7 +17,11 @@ import { healthRoutes } from './routes/health.js'
 import { controlRoutes } from './routes/control.js'
 import { auditRoutes } from './routes/audit.js'
 import { agentRoutes } from './routes/agent.js'
+import { enrollmentRoutes } from './routes/enrollment.js'
+import { policyRolloutsRoutes } from './routes/policy-rollouts.js'
+import { webhooksRoutes } from './routes/webhooks.js'
 import { refreshAllLists } from './services/lists/refresh.js'
+import { runStatsRetention } from './services/analytics.js'
 import { repos } from '@evofw/db'
 import {
   isValidInstallSlug,
@@ -32,6 +37,7 @@ export async function buildApp(opts: BuildAppOptions = {}) {
   const config = opts.config ?? loadConfig()
 
   const app = Fastify({
+    trustProxy: config.trustProxy ?? false,
     logger: { level: config.logLevel },
     genReqId: () => crypto.randomUUID(),
   }).withTypeProvider<ZodTypeProvider>()
@@ -66,14 +72,11 @@ export async function buildApp(opts: BuildAppOptions = {}) {
           }
         : false,
   })
-  await app.register(import('@fastify/rate-limit'), {
-    max: 300,
-    timeWindow: '1 minute',
-  })
   await app.register(corsPlugin, { config })
   await app.register(errorHandlerPlugin)
   await app.register(dbPlugin, { config, memory: opts.memory })
   await app.register(authPlugin, { config })
+  await app.register(operationalPlugin, { config, memory: opts.memory })
 
   // Seed enroll_seed into settings if empty
   if (!repos.getSetting(app.db, 'enroll_seed')) {
@@ -82,12 +85,15 @@ export async function buildApp(opts: BuildAppOptions = {}) {
 
   await app.register(healthRoutes)
   await app.register(agentRoutes, { config })
+  await app.register(enrollmentRoutes, { config })
 
   await app.register(
     async (protectedApi) => {
       protectedApi.addHook('onRequest', app.requireAuth)
       await protectedApi.register(controlRoutes, { config })
       await protectedApi.register(auditRoutes)
+      await protectedApi.register(policyRolloutsRoutes, { config })
+      await protectedApi.register(webhooksRoutes, { config, memory: opts.memory })
     },
     { prefix: '/api/v1' },
   )
@@ -102,6 +108,9 @@ export async function buildApp(opts: BuildAppOptions = {}) {
 
   app.setNotFoundHandler(async (request, reply) => {
     const path = request.url.split('?')[0] ?? ''
+    if (path === '/api' || path.startsWith('/api/') || path === '/v1' || path.startsWith('/v1/')) {
+      return reply.code(404).send({ error: { code: 'NOT_FOUND', message: 'Not Found', request_id: request.id } })
+    }
     const segment = path.startsWith('/') ? path.slice(1) : path
     if (
       request.method === 'GET' &&
@@ -117,7 +126,7 @@ export async function buildApp(opts: BuildAppOptions = {}) {
           config.publicBaseUrl,
           config.enrollSeed,
         )
-        return reply.type(contentType).send(body)
+        return reply.header('Cache-Control', 'no-store').type(contentType).send(body)
       }
     }
 
@@ -149,31 +158,20 @@ export async function buildApp(opts: BuildAppOptions = {}) {
 
     // Raw stats samples grow ~1 row per agent per sync interval; drop old
     // ones daily. Lifetime totals live on agents; aggregates are kept.
-    const runRetention = async () => {
+    const retentionAbort = new AbortController()
-      const cutoff = new Date(
+    let retentionRunning: Promise<void> | null = null
-        Date.now() - config.statsRetentionDays * 86_400_000,
+    const runRetention = () => {
-      ).toISOString()
+      if (retentionRunning) return retentionRunning
-      const deleted = repos.deleteStatsSamplesBefore(app.db, cutoff)
+      retentionRunning = runStatsRetention(app.db, new Date().toISOString(), 1000, retentionAbort.signal, {
-      if (deleted > 0) {
+        onProgress: (progress) => { if (progress.processed || progress.deleted || progress.pending) app.log.info(progress, 'bounded stats maintenance completed') },
-        app.log.info(
+      }).then(() => {}).catch(err => { app.log.warn({ err }, 'stats retention failed') }).finally(() => { retentionRunning = null })
-          { deleted, cutoff },
+      return retentionRunning
-          `stats retention: raw samples older than ${config.statsRetentionDays}d removed`,
-        )
     }
-    }
+    // Delay startup work, then resume bounded durable batches. One promise owns
-    app.scheduler.addCronJob(
+    // every tick and shutdown, so an initial run can never overlap a later tick.
-      new CronJob(
+    const retentionTimer = setInterval(() => { void runRetention() }, 30_000)
-        { cronExpression: '17 3 * * *' },
+    retentionTimer.unref()
-        new AsyncTask('stats-retention', runRetention, (err) => {
+    app.addHook('onClose', async () => { clearInterval(retentionTimer); retentionAbort.abort(); await retentionRunning })
-          app.log.warn({ err }, 'stats retention failed')
-        }),
-        { preventOverrun: true },
-      ),
-    )
-    // First cleanup right at startup, not only after the next 03:17.
-    runRetention().catch((err) =>
-      app.log.warn({ err }, 'stats retention failed'),
-    )
   }
 
   return app

apps/api/src/config.ts

@@ -1,4 +1,5 @@
 import { resolve } from 'node:path'
+import { isIP } from 'node:net'
 
 export interface AppConfig {
   databaseUrl: string
@@ -16,6 +17,7 @@ export interface AppConfig {
   corsOrigins: string[]
   secretKey: string | null
   statsRetentionDays: number
+  trustProxy?: false | string[]
 }
 
 function boolEnv(v: string | undefined, fallback: boolean): boolean {
@@ -26,6 +28,18 @@ function boolEnv(v: string | undefined, fallback: boolean): boolean {
 const DEV_JWT_SECRET = 'dev-secret-change-me'
 const DEV_ENROLL_SEED = 'dev-enroll-seed-change-me'
 
+export function trustedProxyCidrs(value: string | undefined): false | string[] {
+  const entries = (value || '').split(',').map(s => s.trim()).filter(Boolean)
+  for (const entry of entries) {
+    const [address, mask, extra] = entry.split('/')
+    const family = isIP(address!)
+    if (!family || extra !== undefined || (mask !== undefined && (!/^\d+$/.test(mask) || Number(mask) > (family === 4 ? 32 : 128)))) {
+      throw new Error('TRUST_PROXY_CIDRS must contain explicit IP addresses or CIDRs')
+    }
+  }
+  return entries.length ? entries : false
+}
+
 export function loadConfig(): AppConfig {
   const isProd = process.env.NODE_ENV === 'production'
   const jwtSecret =
@@ -70,6 +84,7 @@ export function loadConfig(): AppConfig {
       1,
       Math.min(3650, Number(process.env.STATS_RETENTION_DAYS ?? '30') || 30),
     ),
+    trustProxy: trustedProxyCidrs(process.env.TRUST_PROXY_CIDRS),
   }
 
   // Fail-safe: a production process must not start wide open or with

apps/api/src/plugins/auth.ts

@@ -8,6 +8,7 @@ import {
 import type { AppConfig } from '../config.js'
 import { createHash } from 'node:crypto'
 import { repos } from '@evofw/db'
+import { authenticateTokenExchange } from '../services/enrollment.js'
 
 declare module 'fastify' {
   interface FastifyRequest {
@@ -61,6 +62,8 @@ function isAgentPath(url: string): boolean {
     path === '/v1/agent/policy' ||
     path === '/v1/agent/policy.rsc' ||
     path === '/v1/agent/apply-report' ||
+    path === '/v1/agent/credentials' ||
+    path === '/v1/agent/token-rotation' ||
     path === '/v1/agent/heartbeat'
   )
 }
@@ -122,12 +125,16 @@ async function authPlugin(
       try {
         await request.jwtVerify()
       } catch {
+        if (!app.allowFailedAuthAttempt(request.ip)) return reply.code(429).header('Retry-After', '60').send({ error: { code: 'RATE_LIMITED', message: 'Too many authentication failures' } })
         return reply.code(401).send({
           error: { code: 'UNAUTHORIZED', message: 'Требуется авторизация' },
         })
       }
 
       const payload = request.user
+      if (typeof payload.sub !== 'string' || !payload.sub || payload.sub.length > 256) {
+        return reply.code(401).send({ error: { code: 'UNAUTHORIZED', message: 'Invalid user identity' } })
+      }
       const apps = Array.isArray(payload.apps)
         ? payload.apps.map(String)
         : []
@@ -135,7 +142,7 @@ async function authPlugin(
         ? payload.permissions.map(String)
         : []
 
-      if (!apps.includes('fw') && !payload.is_admin) {
+      if (!apps.includes('fw') && payload.is_admin !== true) {
         return reply.code(403).send({
           error: {
             code: 'FORBIDDEN',
@@ -149,7 +156,7 @@ async function authPlugin(
         email: String(payload.email ?? ''),
         name: String(payload.name ?? ''),
         apps,
-        permissions: payload.is_admin
+        permissions: payload.is_admin === true
           ? [
               'fw:dashboard:read',
               'fw:agents:write',
@@ -160,10 +167,10 @@ async function authPlugin(
               'fw:audit:read',
             ]
           : permissions,
-        isAdmin: Boolean(payload.is_admin),
+        isAdmin: payload.is_admin === true,
       }
 
-      const required = permissionForRequest(request.method, request.url)
+      const required = request.routeOptions.config.requiredPermission ?? permissionForRequest(request.method, request.url)
       if (
         required &&
         !request.authUser.isAdmin &&
@@ -185,13 +192,24 @@ async function authPlugin(
     if (isAgentPath(request.url)) {
       const auth = request.headers.authorization
       if (!auth?.startsWith('Bearer ')) {
+        if (!app.allowFailedAuthAttempt(request.ip)) return reply.code(429).header('Retry-After', '60').send({ error: { code: 'RATE_LIMITED', message: 'Too many authentication failures' } })
         return reply.code(401).send({
           error: { code: 'UNAUTHORIZED', message: 'Agent token required' },
         })
       }
       const token = auth.slice('Bearer '.length).trim()
+      if (request.method === 'POST' && request.url.split('?')[0] === '/v1/agent/token-rotation') {
+        const agentId = authenticateTokenExchange(app.db, app.sqlite, token)
+        if (!agentId) {
+          if (!app.allowFailedAuthAttempt(request.ip)) return reply.code(429).header('Retry-After', '60').send({ error: { code: 'RATE_LIMITED', message: 'Too many authentication failures' } })
+          return reply.code(401).send({ error: { code: 'UNAUTHORIZED', message: 'Invalid agent token' } })
+        }
+        request.agentId = agentId
+        return
+      }
       const agent = repos.getAgentByTokenHash(app.db, hashToken(token))
       if (!agent || agent.status !== 'approved') {
+        if (!app.allowFailedAuthAttempt(request.ip)) return reply.code(429).header('Retry-After', '60').send({ error: { code: 'RATE_LIMITED', message: 'Too many authentication failures' } })
         return reply.code(agent?.status === 'pending' ? 403 : 401).send({
           error: {
             code: agent?.status === 'pending' ? 'FORBIDDEN' : 'UNAUTHORIZED',

apps/api/src/plugins/error-handler.ts

@@ -1,6 +1,7 @@
 import type { FastifyInstance } from 'fastify'
 import fp from 'fastify-plugin'
 import { ZodError } from 'zod'
+import { PolicyPinnedError } from '@evofw/db'
 
 export class AppError extends Error {
   constructor(
@@ -15,7 +16,7 @@ export class AppError extends Error {
 
 async function errorHandlerPlugin(app: FastifyInstance) {
   app.setErrorHandler((err, req, reply) => {
-    if (err instanceof AppError) {
+    if (err instanceof AppError || err instanceof PolicyPinnedError) {
       return reply.code(err.statusCode).send({
         error: {
           code: err.code,

apps/api/src/plugins/operational.ts

@@ -0,0 +1,105 @@
+import fp from 'fastify-plugin'
+import type { FastifyPluginAsync } from 'fastify'
+import { hasPermission, permissionForRequest } from '@evofw/shared'
+import { stat, statfs } from 'node:fs/promises'
+import { dirname } from 'node:path'
+import type { AppConfig } from '../config.js'
+import { AuditOutbox, AUDIT_QUEUE_LIMIT } from '../services/audit-outbox.js'
+
+type PermissionPolicy = 'public' | 'agent' | 'protected' | 'section-filtered'
+export interface RouteSecurity { method: string; url: string; policy: PermissionPolicy; permission: string | null }
+declare module 'fastify' {
+  interface FastifyContextConfig { permissionPolicy?: PermissionPolicy; requiredPermission?: string | null }
+  interface FastifyInstance { auditOutbox: AuditOutbox; routeSecurity: RouteSecurity[]; allowFailedAuthAttempt: (ip: string) => boolean }
+}
+
+export function routeSecurityFor(method: string, url: string): RouteSecurity {
+  const path = url.split('?')[0]!
+  if (['/v1/agent/policy', '/v1/agent/policy.rsc', '/v1/agent/apply-report', '/v1/agent/heartbeat', '/v1/agent/credentials', '/v1/agent/token-rotation'].includes(path)) return { method, url, policy: 'agent', permission: null }
+  const read = ['GET', 'HEAD'].includes(method)
+  const publicAgent = (path === '/v1/agent/enroll' && method === 'POST') || (read && ['/v1/agent/install.sh', '/v1/agent/uninstall.sh', '/v1/agent/sync-script', '/v1/agent/mikrotik-install.rsc'].includes(path))
+  if (publicAgent || (read && ['/api/v1/auth/config', '/api/auth/config'].includes(path))) return { method, url, policy: 'public', permission: null }
+  if (path === '/v1' || path.startsWith('/v1/')) throw new Error(`Agent route lacks authentication mapping: ${method} ${path}`)
+  if (path !== '/api' && !path.startsWith('/api/')) return { method, url, policy: 'public', permission: null }
+  if (path === '/api/v1/ui/summary' && ['GET', 'HEAD'].includes(method)) return { method, url, policy: 'section-filtered', permission: null }
+  const permission = permissionForRequest(method, path)
+  if (!permission) throw new Error(`Protected route lacks permission mapping: ${method} ${path}`)
+  return { method, url, policy: 'protected', permission }
+}
+
+export class IdentityLimiter {
+  private buckets = new Map<string, { start: number; count: number }>()
+  rejected = 0
+  constructor(readonly capacity = 10_000) {}
+  take(key: string, limit: number, now = Date.now()): boolean {
+    let value = this.buckets.get(key)
+    if (!value || now - value.start >= 60_000) {
+      if (!value && this.buckets.size >= this.capacity) {
+        for (const [id, bucket] of this.buckets) if (now - bucket.start >= 60_000) this.buckets.delete(id)
+        if (this.buckets.size >= this.capacity) { this.rejected++; return false }
+      }
+      value = { start: now, count: 0 }; this.buckets.set(key, value)
+    }
+    if (value.count >= limit) { this.rejected++; return false }
+    value.count++; return true
+  }
+  stats() { return { buckets: this.buckets.size, capacity: this.capacity, rejected: this.rejected } }
+  available(key: string, limit: number, now = Date.now()): boolean {
+    const bucket = this.buckets.get(key)
+    return !bucket || now - bucket.start >= 60_000 || bucket.count < limit
+  }
+}
+
+const operationalPlugin: FastifyPluginAsync<{ config: AppConfig; memory?: boolean }> = async (app, { config, memory }) => {
+  const limiter = new IdentityLimiter()
+  const failures = new IdentityLimiter()
+  const queue = new AuditOutbox(app.sqlite, config)
+  app.decorate('auditOutbox', queue)
+  app.decorate('routeSecurity', [])
+  app.decorate('allowFailedAuthAttempt', (ip: string) => failures.available(ip, 60))
+  app.addHook('onResponse', async (request, reply) => {
+    if (reply.statusCode === 401 || reply.statusCode === 403) failures.take(request.ip, 60)
+  })
+  app.addHook('onRoute', route => {
+    for (const method of Array.isArray(route.method) ? route.method : [route.method]) {
+      const security = routeSecurityFor(method, route.url)
+      app.routeSecurity.push(security)
+      route.config = { ...route.config, permissionPolicy: security.policy, requiredPermission: security.permission }
+    }
+  })
+  app.addHook('preHandler', async (request, reply) => {
+    const security = request.routeOptions.config
+    if (security.permissionPolicy === 'agent' && !request.agentId) return reply.code(401).send({ error: { code: 'UNAUTHORIZED', message: 'Agent token required' } })
+    if (security.permissionPolicy === 'protected' || security.permissionPolicy === 'section-filtered') {
+      if (!request.authUser) await app.requireAuth(request, reply)
+      if (reply.sent) return reply
+      if (security.requiredPermission && !request.authUser?.isAdmin && !hasPermission(request.authUser?.permissions || [], security.requiredPermission)) {
+        return reply.code(403).send({ error: { code: 'FORBIDDEN', message: 'Недостаточно прав', request_id: request.id } })
+      }
+    }
+    const path = request.url.split('?')[0]!
+    if (path === '/health' || path === '/ready' || (!path.startsWith('/api/') && !path.startsWith('/v1/'))) return
+    const enrollment = path === '/v1/agent/enroll'
+    const key = enrollment ? `enroll:${request.ip}` : request.agentId ? `agent:${request.agentId}` : request.authUser ? `user:${request.authUser.id}` : `public:${request.ip}`
+    const limit = enrollment ? 20 : request.agentId ? 120 : request.authUser ? 600 : 120
+    if (!limiter.take(key, limit)) return reply.code(429).header('Retry-After', '60').send({ error: { code: 'RATE_LIMITED', message: 'Too many requests', request_id: request.id } })
+    if (queue.enabled && security.permissionPolicy === 'protected' && !['GET', 'HEAD', 'OPTIONS'].includes(request.method) && queue.size() >= AUDIT_QUEUE_LIMIT) {
+      return reply.code(503).send({ error: { code: 'AUDIT_BACKPRESSURE', message: 'Audit delivery queue is full', request_id: request.id } })
+    }
+  })
+  app.get('/api/v1/diagnostics', async () => {
+    const path = config.databaseUrl.replace(/^sqlite:/, '')
+    const fileSize = async (name: string) => { try { return (await stat(name)).size } catch { return null } }
+    let disk: { free_bytes: number; total_bytes: number } | null = null
+    if (!memory) try { const data = await statfs(dirname(path)); disk = { free_bytes: data.bavail * data.bsize, total_bytes: data.blocks * data.bsize } } catch { /* exposed as unavailable */ }
+    return {
+      database: { memory: !!memory, db_bytes: memory ? null : await fileSize(path), wal_bytes: memory ? null : await fileSize(`${path}-wal`), disk },
+      audit_outbox: queue.stats(), limiter: limiter.stats(), auth_failures: failures.stats(),
+      process: { uptime_seconds: process.uptime(), rss_bytes: process.memoryUsage().rss, heap_used_bytes: process.memoryUsage().heapUsed },
+    }
+  })
+  const timer = memory ? null : setInterval(() => { void queue.flush().catch(() => app.log.warn('audit outbox worker failed')) }, 5000)
+  timer?.unref()
+  app.addHook('onClose', async () => { if (timer) clearInterval(timer); await queue.close() })
+}
+export default fp(operationalPlugin, { name: 'operational', dependencies: ['auth', 'db'] })

apps/api/src/routes/agent-labels.ts

@@ -0,0 +1,46 @@
+import type { FastifyPluginAsync } from 'fastify'
+import { repos, type AgentLabelRow } from '@evofw/db'
+import { agentLabelBodySchema, patchAgentLabelSchema, putAgentLabelsSchema, previewLabelAssignmentSchema, applyLabelAssignmentSchema, policyPageSchema } from '@evofw/shared'
+import type { AppConfig } from '../config.js'
+import { AppError } from '../plugins/error-handler.js'
+import { auditMutation } from '../services/audit.js'
+import { createAgentLabel, patchAgentLabel, putAgentLabels, previewLabelAssignment, applyLabelAssignment } from '../services/labels.js'
+
+export const agentLabelsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (app, { config }) => {
+  const map = (row: AgentLabelRow) => { const { name_key: _, ...item } = row; return item }
+  const audit = (req: Parameters<typeof auditMutation>[2], action: string, id: string, details?: Record<string, unknown>) => auditMutation(app, config, req, { action, targetType: 'app_resource', targetId: id, summary: action, details })
+  app.get('/agent-labels', async (req) => {
+    const page = policyPageSchema.parse(req.query)
+    return { items: repos.listAgentLabels(app.db, page.limit, page.offset).map(map), total: repos.countAgentLabels(app.db) }
+  })
+  app.post('/agent-labels', async (req, reply) => {
+    const row = createAgentLabel(app.db, agentLabelBodySchema.parse(req.body))
+    audit(req, 'agent_label.create', row.id)
+    return reply.code(201).send(map(row))
+  })
+  app.patch<{ Params: { id: string } }>('/agent-labels/:id', async (req) => {
+    const row = patchAgentLabel(app.db, req.params.id, patchAgentLabelSchema.parse(req.body))
+    audit(req, 'agent_label.update', row.id)
+    return map(row)
+  })
+  app.delete<{ Params: { id: string } }>('/agent-labels/:id', async (req) => {
+    if (!repos.deleteAgentLabel(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Label not found', 404)
+    audit(req, 'agent_label.delete', req.params.id)
+    return { ok: true }
+  })
+  app.get<{ Params: { id: string } }>('/agents/:id/labels', async (req) => {
+    if (!repos.getAgent(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Agent not found', 404)
+    return { agent_id: req.params.id, label_ids: repos.agentLabelIds(app.db, req.params.id) }
+  })
+  app.put<{ Params: { id: string } }>('/agents/:id/labels', async (req) => {
+    const result = putAgentLabels(app.db, req.params.id, putAgentLabelsSchema.parse(req.body).label_ids)
+    audit(req, 'agent.labels.update', req.params.id, { label_ids: result.label_ids })
+    return result
+  })
+  app.post('/agent-labels/preview-assignment', async (req) => previewLabelAssignment(app.db, previewLabelAssignmentSchema.parse(req.body)))
+  app.post('/agent-labels/assign-policy-sets', async (req) => {
+    const result = applyLabelAssignment(app.db, applyLabelAssignmentSchema.parse(req.body))
+    audit(req, 'agent_label.assign_policy_sets', result.items[0]!.agent_id, { affected_agent_ids: result.affected_agent_ids })
+    return result
+  })
+}

apps/api/src/routes/agent.ts

@@ -1,13 +1,13 @@
 import { readFileSync } from 'node:fs'
-import { createHash, timingSafeEqual } from 'node:crypto'
+import { createHash } from 'node:crypto'
 import { join } from 'node:path'
 import type { FastifyPluginAsync } from 'fastify'
 import { repos } from '@evofw/db'
-import { enrollBodySchema, applyReportBodySchema, hostFwRuleSchema, hostListenerSchema } from '@evofw/shared'
+import { applyReportBodySchema, POLICY_BUDGET, POLICY_BUDGET_ERROR, agentPolicyQuerySchema, policyBudgetErrorEnvelopeSchema } from '@evofw/shared'
 import type { AppConfig } from '../config.js'
-import { hashToken } from '../plugins/auth.js'
+import { evaluateAgentPolicy, policyBudgetExceeded } from '../services/policy/evaluate.js'
-import { evaluateAgentPolicy } from '../services/policy/evaluate.js'
 import { renderMikrotikPolicyRsc } from '../services/policy/mikrotik-rsc.js'
+import { ingestTelemetry } from '../services/telemetry.js'
 import { AppError } from '../plugins/error-handler.js'
 import { resolveAgentScriptsDir } from '../services/agent-scripts-path.js'
 import { resolveAndRenderInstall } from '../services/install-links.js'
@@ -18,33 +18,8 @@ import {
 
 const scriptsDir = resolveAgentScriptsDir()
 
-/** Constant-time seed check; hash first so lengths always match. */
-function seedMatches(presented: string | undefined, expected: string): boolean {
-  if (!presented) return false
-  const a = createHash('sha256').update(presented).digest()
-  const b = createHash('sha256').update(expected).digest()
-  return timingSafeEqual(a, b)
-}
 
-function sanitizeHostFirewall(raw: {
+
-  rules?: unknown[]
-  listeners?: unknown[]
-}): { rules: unknown[]; listeners: unknown[] } {
-  const rules: unknown[] = []
-  for (const item of raw.rules ?? []) {
-    const parsed = hostFwRuleSchema.safeParse(item)
-    if (parsed.success) rules.push(parsed.data)
-  }
-  const listeners: unknown[] = []
-  for (const item of raw.listeners ?? []) {
-    const parsed = hostListenerSchema.safeParse(item)
-    if (parsed.success) listeners.push(parsed.data)
-  }
-  return {
-    rules: rules.slice(0, 500),
-    listeners: listeners.slice(0, 200),
-  }
-}
 
 export const agentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   app,
@@ -68,7 +43,7 @@ export const agentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
         config.publicBaseUrl,
         config.enrollSeed,
       )
-      return reply.type(contentType).send(body)
+      return reply.header('cache-control', 'no-store').type(contentType).send(body)
     },
   )
 
@@ -96,88 +71,10 @@ export const agentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
     return reply.type('text/plain').send(body)
   })
 
-  app.post('/v1/agent/enroll', async (req, reply) => {
-    const seed = req.headers['x-evofw-seed']
-    const expected =
-      repos.getSetting(app.db, 'enroll_seed') || config.enrollSeed
-    if (!seedMatches(String(seed ?? ''), expected)) {
-      throw new AppError('UNAUTHORIZED', 'Invalid enroll seed', 401)
-    }
-    const body = enrollBodySchema.parse(req.body)
-    const tokenHash = hashToken(body.token)
-    const existingByToken = repos.getAgentByTokenHash(app.db, tokenHash)
-    if (existingByToken) {
-      throw new AppError('CONFLICT', 'Token already enrolled', 409)
-    }
 
-    if (body.install_link_id) {
+  app.get('/v1/agent/policy', async (req, reply) => {
-      const link = repos.getInstallLink(app.db, body.install_link_id)
+    const query = agentPolicyQuerySchema.parse(req.query)
-      if (!link) {
+    try {
-        throw new AppError('NOT_FOUND', 'Install link not found', 404)
-      }
-      if (link.revokedAt) {
-        throw new AppError('GONE', 'Install link revoked', 410)
-      }
-      if (!link.agentId) {
-        throw new AppError('CONFLICT', 'Install link has no agent', 409)
-      }
-      const invited = repos.getAgent(app.db, link.agentId)
-      if (!invited) {
-        throw new AppError('NOT_FOUND', 'Agent not found', 404)
-      }
-      if (invited.status !== 'invited' && invited.status !== 'pending') {
-        throw new AppError(
-          'CONFLICT',
-          `Agent status is ${invited.status}`,
-          409,
-        )
-      }
-      const agent = repos.updateAgent(app.db, invited.id, {
-        name: body.name,
-        hostname: body.hostname ?? null,
-        platform: body.platform ?? invited.platform,
-        tokenPrefix: body.token.slice(0, 12),
-        tokenHash,
-        status: 'pending',
-        clientVersion: body.client_version ?? null,
-        lastSeenAt: new Date().toISOString(),
-        lastSeenIp: req.ip,
-      })
-      return reply.code(201).send({
-        client_id: agent!.id,
-        id: agent!.id,
-        status: agent!.status,
-        name: agent!.name,
-      })
-    }
-
-    const id = crypto.randomUUID()
-    const now = new Date().toISOString()
-    const agent = repos.insertAgent(app.db, {
-      id,
-      name: body.name,
-      hostname: body.hostname ?? null,
-      platform: body.platform ?? 'linux',
-      tokenPrefix: body.token.slice(0, 12),
-      tokenHash,
-      status: 'pending',
-      defaultAction: 'accept',
-      policyGeneration: 1,
-      clientVersion: body.client_version ?? null,
-      settingsJson: '{}',
-      createdAt: now,
-      lastSeenAt: now,
-      lastSeenIp: req.ip,
-    })
-    return reply.code(201).send({
-      client_id: agent!.id,
-      id: agent!.id,
-      status: agent!.status,
-      name: agent!.name,
-    })
-  })
-
-  app.get('/v1/agent/policy', async (req) => {
       const agentId = req.agentId!
       // Record contact first — empty policy (no rule sets) is a valid online state.
       repos.updateAgent(app.db, agentId, {
@@ -186,7 +83,7 @@ export const agentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
       })
       const policy = evaluateAgentPolicy(app.db, agentId)
       const scriptSha = getSyncScriptMeta().sha256
-    return {
+      const response = {
         generation: policy.generation,
         hash: policy.hash,
         apply_version: policy.applyVersion,
@@ -214,6 +111,20 @@ export const agentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
             ? policy.denyCidrs.length
             : policy.allowCidrs.length,
       }
+      const json = JSON.stringify(response)
+      if (Buffer.byteLength(json) > POLICY_BUDGET.jsonBytes) policyBudgetExceeded('8MiB policy response JSON')
+      const etag = `"policy-v1-${createHash('sha256').update(json).digest('hex')}"`
+      repos.updateAgent(app.db, agentId, { desiredHash: policy.hash, desiredHashGeneration: policy.generation, lastPolicyDeliveredAt: new Date().toISOString() })
+      reply.header('etag', etag).header('cache-control', 'private, no-cache')
+      if (ifNoneMatchHits(req.headers['if-none-match'], etag)) return reply.code(304).send()
+      return reply.type('application/json').send(json)
+    } catch (error) {
+      if (query.transport === 'routeros' && error instanceof AppError && error.code === POLICY_BUDGET_ERROR) {
+        reply.removeHeader('etag')
+        return reply.header('cache-control', 'no-store').code(200).send(policyBudgetErrorEnvelopeSchema.parse({ error: { code: POLICY_BUDGET_ERROR, message: error.message.slice(0, 512) } }))
+      }
+      throw error
+    }
   })
 
   app.get('/v1/agent/policy.rsc', async (req, reply) => {
@@ -223,98 +134,19 @@ export const agentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
       lastSeenIp: req.ip,
     })
     const policy = evaluateAgentPolicy(app.db, agentId)
-    return reply.type('text/plain').send(renderMikrotikPolicyRsc(policy))
+    const script = renderMikrotikPolicyRsc(policy)
+    if (Buffer.byteLength(script) > POLICY_BUDGET.jsonBytes) policyBudgetExceeded('8MiB RouterOS policy response')
+    repos.updateAgent(app.db, agentId, { desiredHash: policy.hash, desiredHashGeneration: policy.generation, lastPolicyDeliveredAt: new Date().toISOString() })
+    const etag = `"policy-rsc-v1-${createHash('sha256').update(script).digest('hex')}"`
+    reply.header('etag', etag).header('cache-control', 'private, no-cache')
+    if (ifNoneMatchHits(req.headers['if-none-match'], etag)) return reply.code(304).send()
+    return reply.type('text/plain').send(script)
   })
 
   app.post('/v1/agent/apply-report', async (req) => {
-    const agentId = req.agentId!
     const body = applyReportBodySchema.parse(req.body)
-    const now = new Date().toISOString()
+    const result = ingestTelemetry(app.db, req.agentId!, body, req.ip)
-    const prev = repos.getAgent(app.db, agentId)
+    return { ...result, ...(body.telemetry_version === 2 ? { report_id: body.report_id, sequence: body.sequence, sequence_floor: repos.getTelemetryState(app.db, req.agentId!)!.sequence } : {}) }
-    const reportedDropped = body.packets_dropped ?? 0
-    const reportedAccepted = body.packets_accepted ?? 0
-    const prevDropped = prev?.lastApplyPacketsDropped ?? 0
-    const prevAccepted = prev?.lastApplyPacketsAccepted ?? 0
-    // Absolute-since-chain-create from agent. If counters reset (re-apply),
-    // treat the new absolute as the delta; else add the increase.
-    const deltaDropped =
-      reportedDropped >= prevDropped
-        ? reportedDropped - prevDropped
-        : reportedDropped
-    const deltaAccepted =
-      reportedAccepted >= prevAccepted
-        ? reportedAccepted - prevAccepted
-        : reportedAccepted
-    const totalDropped = (prev?.totalPacketsDropped ?? 0) + deltaDropped
-    const totalAccepted = (prev?.totalPacketsAccepted ?? 0) + deltaAccepted
-
-    // Chain/set counters were flushed (policy apply). Zero per-IP baselines so
-    // the next epoch of element counters accumulates (zeros are omitted from ip_hits).
-    // Port hits use a separate dynamic set that is not flushed on apply — leave baselines.
-    if (reportedDropped < prevDropped) {
-      repos.resetIpBlockStatsBaselines(app.db, agentId)
-    }
-
-    repos.updateAgent(app.db, agentId, {
-      lastApplyAt: now,
-      lastApplyStatus: body.status,
-      lastApplyError: body.error ?? null,
-      lastApplyPrefixCount: body.prefix_count ?? 0,
-      lastApplyPacketsDropped: reportedDropped,
-      lastApplyPacketsAccepted: reportedAccepted,
-      totalPacketsDropped: totalDropped,
-      totalPacketsAccepted: totalAccepted,
-      lastApplyKernelMethod: body.kernel_method ?? null,
-      lastSeenAt: now,
-      lastSeenIp: req.ip,
-    })
-    repos.insertStatsSample(app.db, {
-      id: crypto.randomUUID(),
-      agentId,
-      packetsDropped: reportedDropped,
-      packetsAccepted: reportedAccepted,
-      prefixCount: body.prefix_count ?? 0,
-      kernelMethod: body.kernel_method ?? null,
-      recordedAt: now,
-    })
-    if (body.ip_hits?.length) {
-      const presence = body.source === 'mikrotik'
-      repos.upsertIpBlockStats(app.db, agentId, body.ip_hits, now, {
-        mode: presence ? 'presence' : 'absolute',
-      })
-    }
-    if (body.port_hits?.length && body.source !== 'mikrotik') {
-      repos.upsertPortBlockStats(app.db, agentId, body.port_hits, now)
-    }
-    if (body.host_firewall && body.source !== 'mikrotik') {
-      const sanitized = sanitizeHostFirewall(body.host_firewall)
-      // Empty snapshot must not wipe a previous good collect (agent bash bugs / race).
-      if (
-        sanitized.rules.length === 0 &&
-        sanitized.listeners.length === 0
-      ) {
-        const existing = repos.getHostFirewallSnapshot(app.db, agentId)
-        if (existing) {
-          return { ok: true }
-        }
-      }
-      const payloadJson = JSON.stringify({
-        rules: sanitized.rules,
-        listeners: sanitized.listeners,
-      })
-      const rawDigest = createHash('sha256')
-        .update(payloadJson)
-        .digest('hex')
-        .slice(0, 16)
-      repos.upsertHostFirewallSnapshot(
-        app.db,
-        agentId,
-        payloadJson,
-        now,
-        rawDigest,
-      )
-    }
-    return { ok: true }
   })
 
   app.post('/v1/agent/heartbeat', async (req) => {

apps/api/src/routes/agents.ts

@@ -6,13 +6,16 @@ import {
   patchAgentBodySchema,
   cloneFromBodySchema,
   agentIdsBodySchema,
+  explainQuerySchema,
+  agentLabelFilterSchema,
 } from '@evofw/shared'
 import { AppError } from '../plugins/error-handler.js'
 import { evaluateAgentPolicy, truncateCidrs } from '../services/policy/evaluate.js'
 import { buildInstallUrls } from '../services/install-links.js'
 import type { AppConfig } from '../config.js'
 import { auditMutation } from '../services/audit.js'
-import { mapAgent } from '../services/row-mappers.js'
+import { mapAgent, mapOverride } from '../services/row-mappers.js'
+import { explainAgentPolicy } from '../services/policy/explain.js'
 import { applyPagination } from '../services/pagination.js'
 
 export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
@@ -21,10 +24,17 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
 ) => {
   const { config } = opts
 
-  app.get<{ Querystring: { limit?: string; offset?: string } }>(
+  app.get<{ Params: { id: string }; Querystring: Record<string, unknown> }>('/agents/:id/explain', async (req) => {
+    if (!repos.getAgent(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Agent not found', 404)
+    return explainAgentPolicy(app.db, req.params.id, explainQuerySchema.parse(req.query))
+  })
+
+  app.get<{ Querystring: { limit?: string; offset?: string; label_ids?: string | string[] } }>(
     '/agents',
     async (req) => {
-      const all = repos.listAgents(app.db)
+      const labels = agentLabelFilterSchema.parse(req.query).label_ids
+      const matching = labels?.length ? new Set(repos.agentIdsWithLabels(app.db, labels)) : null
+      const all = repos.listAgents(app.db).filter((agent) => !matching || matching.has(agent.id))
       const linksByAgent = repos.mapActiveInstallLinksByAgentId(app.db)
       const items = all.map((a) => {
         const link = linksByAgent.get(a.id)
@@ -137,11 +147,14 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   app.post<{ Params: { id: string } }>('/agents/:id/approve', async (req) => {
     const a = repos.getAgent(app.db, req.params.id)
     if (!a) throw new AppError('NOT_FOUND', 'Agent not found', 404)
-    const updated = repos.updateAgent(app.db, a.id, {
+    const updated = app.db.transaction((tx) => {
+      repos.updateAgent(tx, a.id, {
         status: 'approved',
         approvedAt: new Date().toISOString(),
       })
-    repos.ensureSharedSetAssigned(app.db, a.id)
+      repos.ensureSharedSetAssigned(tx, a.id)
+      return repos.getAgent(tx, a.id)!
+    })
     auditMutation(app, config, req, {
       action: 'agent.approve',
       targetType: 'app_resource',
@@ -240,12 +253,14 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
     '/agents/:id/clone-from/:sourceId',
     async (req) => {
       const body = cloneFromBodySchema.parse(req.body ?? {})
-      const updated = repos.cloneRulesFrom(
+      if (req.params.id === req.params.sourceId) throw new AppError('VALIDATION_ERROR', 'Cannot clone an agent onto itself', 400)
-        app.db,
+      let updated: ReturnType<typeof repos.cloneRulesFrom>
-        req.params.sourceId,
+      try {
-        req.params.id,
+        updated = repos.cloneRulesFrom(app.db, req.params.sourceId, req.params.id, body.include_overrides ?? false)
-        body.include_overrides ?? false,
+      } catch (err) {
-      )
+        if ((err as { code?: string }).code === 'OVERRIDE_LIMIT') throw new AppError('CONFLICT', (err as Error).message, 409)
+        throw err
+      }
       if (!updated) throw new AppError('NOT_FOUND', 'Agent not found', 404)
       auditMutation(app, config, req, {
         action: 'agent.clone_rules',
@@ -264,16 +279,10 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
 
   app.get<{ Params: { id: string } }>(
     '/agents/:id/overrides',
-    async (req) => ({
+    async (req) => {
-      items: repos.listOverrides(app.db, req.params.id).map((o) => ({
+      if (!repos.getAgent(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Agent not found', 404)
-        id: o.id,
+      return { items: repos.listOverrides(app.db, req.params.id).map((row) => mapOverride(row)) }
-        agent_id: o.agentId,
+    },
-        cidr: o.cidr,
-        action: o.action,
-        comment: o.comment,
-        created_at: o.createdAt,
-      })),
-    }),
   )
 
   app.post<{ Params: { id: string } }>(
@@ -282,16 +291,30 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
       const body = createOverrideBodySchema.parse(req.body)
       const a = repos.getAgent(app.db, req.params.id)
       if (!a) throw new AppError('NOT_FOUND', 'Agent not found', 404)
-      const row = repos.insertOverride(app.db, {
+      const now = new Date().toISOString()
+      const expiresAt = body.ttl_seconds === undefined ? body.expires_at ? new Date(body.expires_at).toISOString() : null : new Date(Date.parse(now) + body.ttl_seconds * 1000).toISOString()
+      if (expiresAt && (expiresAt <= now || Date.parse(expiresAt) - Date.parse(now) > 30 * 86_400_000)) throw new AppError('VALIDATION_ERROR', 'Override expiry must be in the next 30 days', 400)
+      const row = app.db.transaction((tx) => {
+      let inserted: ReturnType<typeof repos.insertOverride>
+      try { inserted = repos.insertOverride(tx, {
         id: crypto.randomUUID(),
         agentId: a.id,
         cidr: body.cidr,
         action: body.action,
         comment: body.comment ?? null,
+        reason: body.reason ?? body.comment ?? null,
+        createdByName: req.authUser?.name?.slice(0, 128) || req.authUser?.email?.slice(0, 128) || null,
+        expiresAt,
         createdByUserId: req.authUser?.id,
-        createdAt: new Date().toISOString(),
+        createdAt: now,
+      }) } catch (err) {
+        if ((err as { code?: string }).code === 'OVERRIDE_LIMIT') throw new AppError('CONFLICT', (err as Error).message, 409)
+        if ((err as { code?: string }).code === 'SQLITE_CONSTRAINT_UNIQUE') throw new AppError('CONFLICT', 'An active override already exists for this network', 409)
+        throw err
+      }
+      repos.bumpAgentGeneration(tx, a.id)
+      return inserted
       })
-      repos.bumpAgentGeneration(app.db, a.id)
       auditMutation(app, config, req, {
         action: 'override.create',
         targetType: 'app_resource',
@@ -302,23 +325,18 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
           agent_id: a.id,
           cidr: body.cidr,
           action: body.action,
+          reason: row!.reason, expires_at: expiresAt,
         },
       })
-      return {
+      return mapOverride(row!)
-        id: row!.id,
-        agent_id: row!.agentId,
-        cidr: row!.cidr,
-        action: row!.action,
-        comment: row!.comment,
-        created_at: row!.createdAt,
-      }
     },
   )
 
   app.delete<{ Params: { id: string; overrideId: string } }>(
     '/agents/:id/overrides/:overrideId',
     async (req) => {
-      repos.deleteOverride(app.db, req.params.overrideId)
+      const deleted = repos.deleteOverride(app.db, req.params.overrideId, req.params.id)
+      if (!deleted.changes) throw new AppError('NOT_FOUND', 'Override not found for agent', 404)
       repos.bumpAgentGeneration(app.db, req.params.id)
       auditMutation(app, config, req, {
         action: 'override.delete',
@@ -349,6 +367,7 @@ export const agentsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
       try {
         repos.setAgentPolicySets(app.db, a.id, body.set_ids)
       } catch (err) {
+        if ((err as { code?: string }).code === 'POLICY_PINNED') throw err
         throw new AppError(
           'VALIDATION_ERROR',
           err instanceof Error ? err.message : String(err),

apps/api/src/routes/control.ts

@@ -9,7 +9,9 @@ import { rulesRoutes } from './rules.js'
 import { statsRoutes } from './stats.js'
 import { portAclRoutes } from './port-acl.js'
 import { integrationsEvobgpRoutes } from './integrations-evobgp.js'
+import { uiSummaryRoutes } from './ui-summary.js'
 import { settingsRoutes } from './settings.js'
+import { agentLabelsRoutes } from './agent-labels.js'
 
 /** Aggregates domain control-plane route plugins under /api/v1. */
 export const controlRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
@@ -17,9 +19,11 @@ export const controlRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   opts,
 ) => {
   const { config } = opts
+  await app.register(uiSummaryRoutes)
   await app.register(dashboardRoutes, { config })
   await app.register(installLinksRoutes, { config })
   await app.register(agentsRoutes, { config })
+  await app.register(agentLabelsRoutes, { config })
   await app.register(listsRoutes, { config })
   await app.register(policySetsRoutes, { config })
   await app.register(rulesRoutes, { config })

apps/api/src/routes/dashboard.ts

@@ -32,12 +32,11 @@ export const dashboardRoutes: FastifyPluginAsync<{ config: AppConfig }> = async
     }
   })
 
-  app.get('/install-context', async () => {
+  app.get('/install-context', async (_request, reply) => {
-    const seed =
+    reply.header('Cache-Control', 'no-store')
-      repos.getSetting(app.db, 'enroll_seed') || config.enrollSeed
     return {
       suggested_cp_url: config.publicBaseUrl,
-      enroll_seed: seed,
+      enroll_seed_configured: !!(repos.getSetting(app.db, 'enroll_seed') || config.enrollSeed),
       install_sh_url: `${config.publicBaseUrl}/v1/agent/install.sh`,
       mikrotik_url: `${config.publicBaseUrl}/v1/agent/mikrotik-install.rsc`,
       sync_interval_sec: Number(

apps/api/src/routes/enrollment.ts

@@ -0,0 +1,37 @@
+import type { FastifyPluginAsync } from 'fastify'
+import { enrollmentCredentialBodySchema, tokenRotationExchangeBodySchema } from '@evofw/shared'
+import type { AppConfig } from '../config.js'
+import { AppError } from '../plugins/error-handler.js'
+import { enrollWithCredential, exchangeToken, getCredentialStatus, requestTokenRotation, getOperatorCredentialStatus } from '../services/enrollment.js'
+import { auditMutation } from '../services/audit.js'
+
+export const enrollmentRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (app, { config }) => {
+  app.post('/v1/agent/enroll', async (req, reply) => {
+    const body = enrollmentCredentialBodySchema.parse(req.body)
+    const credential = req.headers['x-evofw-enrollment-credential']
+    if (typeof credential !== 'string' || !credential) throw new AppError('UNAUTHORIZED', 'Scoped install credential required', 401)
+    const { created, agent } = enrollWithCredential(app.db, app.sqlite, config.enrollSeed, body, credential, req.ip)
+    reply.header('Cache-Control', 'no-store')
+    return reply.code(created ? 201 : 200).send({ client_id: agent.id, id: agent.id, status: agent.status, name: agent.name })
+  })
+  app.get('/v1/agent/credentials', async (req, reply) => {
+    reply.header('Cache-Control', 'no-store')
+    return getCredentialStatus(app.sqlite, req.agentId!)
+  })
+  app.post('/v1/agent/token-rotation', async (req, reply) => {
+    const auth = req.headers.authorization
+    if (!auth?.startsWith('Bearer ')) throw new AppError('UNAUTHORIZED', 'Agent token required', 401)
+    const result = exchangeToken(app.db, app.sqlite, auth.slice(7).trim(), tokenRotationExchangeBodySchema.parse(req.body))
+    reply.header('Cache-Control', 'no-store')
+    return result
+  })
+  app.get<{ Params: { id: string } }>('/api/v1/agents/:id/token-rotation', { preHandler: app.requireAuth }, async (req, reply) => {
+    reply.header('Cache-Control', 'no-store')
+    return getOperatorCredentialStatus(app.db, app.sqlite, req.params.id)
+  })
+  app.post<{ Params: { id: string } }>('/api/v1/agents/:id/token-rotation', { preHandler: app.requireAuth }, async (req, reply) => {
+    const result = requestTokenRotation(app.db, app.sqlite, req.params.id)
+    auditMutation(app, config, req, { action: 'agent.update', targetType: 'app_resource', targetId: req.params.id, summary: 'Requested agent token rotation', details: { request_id: result.request_id } })
+    return reply.code(202).send(result)
+  })
+}

apps/api/src/routes/install-links.ts

@@ -30,8 +30,8 @@ export const installLinksRoutes: FastifyPluginAsync<{ config: AppConfig }> = asy
 
   app.post('/install-links', async (req, reply) => {
     const body = createInstallLinkBodySchema.parse(req.body)
-    const linkId = randomToken(10)
+    const linkId = randomToken(32)
-    const slug = randomToken(16)
+    const slug = randomToken(32)
     if (
       repos.getInstallLink(app.db, linkId) ||
       repos.getInstallLinkBySlug(app.db, slug)
@@ -67,6 +67,7 @@ export const installLinksRoutes: FastifyPluginAsync<{ config: AppConfig }> = asy
         platform,
         agentId,
         createdAt: now,
+        expiresAt: new Date(Date.now() + 24 * 60 * 60 * 1000).toISOString(),
         useCount: 0,
       })
     })()

apps/api/src/routes/integrations-evobgp.ts

@@ -1,12 +1,14 @@
 import type { FastifyPluginAsync } from 'fastify'
 import { repos } from '@evofw/db'
 import { AppError } from '../plugins/error-handler.js'
+import { decryptSecret } from '../services/secret-cipher.js'
+import { fetchEvobgpPages } from '../services/evobgp.js'
 
 export const integrationsEvobgpRoutes: FastifyPluginAsync = async (app) => {
   /** Proxy EvoBGP communities for UI autocomplete. */
   app.get('/integrations/evobgp/communities', async () => {
     const apiUrl = repos.getSetting(app.db, 'evobgp_api_url')
-    const token = repos.getSetting(app.db, 'evobgp_api_token')
+    const token = decryptSecret(repos.getSetting(app.db, 'evobgp_api_token'))
     if (!apiUrl || !token) {
       throw new AppError(
         'VALIDATION_ERROR',
@@ -15,29 +17,22 @@ export const integrationsEvobgpRoutes: FastifyPluginAsync = async (app) => {
       )
     }
     const base = apiUrl.replace(/\/$/, '')
-    const res = await fetch(`${base}/v1/communities?limit=200`, {
+    let pages: Record<string, unknown>[]
-      headers: {
+    try { pages = await fetchEvobgpPages(base, token, '/v1/communities', 200) } catch (err) {
-        Authorization: `Bearer ${token}`,
+      throw new AppError('UPSTREAM_ERROR', err instanceof Error ? err.message : 'EvoBGP request failed', 502)
-        Accept: 'application/json',
+    }
-      },
+    const all = pages.flatMap((page) => {
-      signal: AbortSignal.timeout(20_000),
+      if (!Array.isArray(page.items)) throw new AppError('UPSTREAM_ERROR', 'Invalid EvoBGP communities response', 502)
+      return page.items.map((item: unknown) => {
+        if (!item || typeof item !== 'object') throw new AppError('UPSTREAM_ERROR', 'Invalid EvoBGP community', 502)
+        const value = item as Record<string, unknown>
+        if (typeof value.id !== 'string' || !value.id || typeof value.community !== 'string' || !value.community || (value.title !== undefined && value.title !== null && typeof value.title !== 'string')) {
+          throw new AppError('UPSTREAM_ERROR', 'Invalid EvoBGP community', 502)
+        }
+        return { id: value.id, community: value.community, title: value.title as string | null | undefined }
       })
-    if (!res.ok) {
+    })
-      throw new AppError(
+    const items = all
-        'UPSTREAM_ERROR',
-        `EvoBGP communities HTTP ${res.status}`,
-        502,
-      )
-    }
-    const data = (await res.json()) as {
-      items?: {
-        id?: string
-        community?: string
-        title?: string | null
-      }[]
-    }
-    const items = (data.items ?? [])
-      .filter((x) => x.id && x.community)
       .map((x) => ({
         id: x.id!,
         community: x.community!,

apps/api/src/routes/lists.ts

@@ -5,29 +5,46 @@ import {
   listEntriesBodySchema,
   deleteListEntryBodySchema,
   isManualListType,
+  listEntryInputSchema,
+  listImportQuerySchema,
+  listDetailPageQuerySchema,
 } from '@evofw/shared'
 import { AppError } from '../plugins/error-handler.js'
 import { refreshIpList } from '../services/lists/refresh.js'
 import {
   addListEntries,
+  addListEntriesSummary,
   deleteListEntry,
+  deleteListEntrySummary,
   mapListDetail,
+  mapListDetailPage,
 } from '../services/lists/entries.js'
 import type { AppConfig } from '../config.js'
 import { auditMutation } from '../services/audit.js'
 import { maskListConfig, sealListConfig } from '../services/secret-cipher.js'
 import { applyPagination } from '../services/pagination.js'
 
+const LIST_IMPORT_BODY_LIMIT = 8 * 1024 * 1024
+const MAX_IMPORT_ENTRIES = 100_000
+function validateImportBudget(values: string[], structuredCount = 0) {
+  let count = structuredCount
+  if (count > MAX_IMPORT_ENTRIES) throw new AppError('VALIDATION_ERROR', 'At most 100000 list entries per import', 400)
+  for (const value of values) for (const _token of value.matchAll(/[^\n,;\s]+/g)) {
+    if (++count > MAX_IMPORT_ENTRIES) throw new AppError('VALIDATION_ERROR', 'At most 100000 list entries per import', 400)
+  }
+}
+
 export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   app,
   opts,
 ) => {
   const { config } = opts
 
-  app.get<{ Querystring: { limit?: string; offset?: string } }>(
+  app.get<{ Querystring: { limit?: string; offset?: string; view?: string } }>(
     '/lists',
     async (req) => {
-      const lists = repos.listIpLists(app.db)
+      if (req.query.view !== undefined && req.query.view !== 'summary') throw new AppError('VALIDATION_ERROR', 'Unknown list catalog view', 400)
+      const lists = req.query.view === 'summary' ? repos.listIpListsSummary(app.db) : repos.listIpLists(app.db)
       const counts = repos.countEntriesByListIds(
         app.db,
         lists.map((l) => l.id),
@@ -36,7 +53,7 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
         id: l.id,
         name: l.name,
         type: l.type,
-        config_json: maskListConfig(l.configJson),
+        config_json: 'configJson' in l && typeof l.configJson === 'string' ? maskListConfig(l.configJson) : '{}',
         content_hash: l.contentHash,
         refreshed_at: l.refreshedAt,
         last_error: l.lastError,
@@ -49,22 +66,37 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
     },
   )
 
-  app.post('/lists', async (req) => {
+  app.post('/lists', { bodyLimit: LIST_IMPORT_BODY_LIMIT }, async (req) => {
     const body = createIpListBodySchema.parse(req.body)
     const type =
       body.type === 'domains' ? 'static' : body.type
+    const inputConfig = { ...(body.config ?? {}) }
+    validateImportBudget(body.entries ?? [], (Array.isArray(inputConfig.items) ? inputConfig.items.length : 0) + (Array.isArray(inputConfig.domains) ? inputConfig.domains.length : 0))
+    if (isManualListType(type) && ((inputConfig.items !== undefined && !Array.isArray(inputConfig.items)) || (inputConfig.domains !== undefined && !Array.isArray(inputConfig.domains)))) {
+      throw new AppError('VALIDATION_ERROR', 'Manual list items/domains must be arrays', 400)
+    }
+    const initialItems = isManualListType(type) && Array.isArray(inputConfig.items)
+      ? inputConfig.items.map((item) => listEntryInputSchema.parse(item))
+      : []
+    const initialDomains = isManualListType(type) && Array.isArray(inputConfig.domains)
+      ? inputConfig.domains.map((domain) => listEntryInputSchema.parse({ kind: 'hostname', value: domain }))
+      : []
+    if (isManualListType(type)) {
+      delete inputConfig.items
+      delete inputConfig.domains
+    }
     const id = crypto.randomUUID()
     const list = repos.insertIpList(app.db, {
       id,
       name: body.name,
       type,
-      configJson: sealListConfig({ ...(body.config ?? {}) }),
+      configJson: sealListConfig(inputConfig),
       createdAt: new Date().toISOString(),
       updatedAt: new Date().toISOString(),
     })
-    if (body.entries?.length && isManualListType(type)) {
+    if (isManualListType(type) && (body.entries?.length || initialItems.length || initialDomains.length)) {
       try {
-        await addListEntries(app.db, id, { values: body.entries })
+        await addListEntries(app.db, id, { values: body.entries, items: [...initialItems, ...initialDomains] })
       } catch (err) {
         repos.deleteIpList(app.db, id)
         throw new AppError(
@@ -94,22 +126,25 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   })
 
   app.get<{ Params: { id: string } }>('/lists/:id', async (req) => {
-    const detail = mapListDetail(app.db, req.params.id)
+    const detail = Object.keys(req.query as object).length > 0
+      ? mapListDetailPage(app.db, req.params.id, listDetailPageQuerySchema.parse(req.query))
+      : mapListDetail(app.db, req.params.id)
     if (!detail) throw new AppError('NOT_FOUND', 'List not found', 404)
     return detail
   })
 
   app.post<{ Params: { id: string } }>(
     '/lists/:id/entries',
+    { bodyLimit: LIST_IMPORT_BODY_LIMIT },
     async (req) => {
       const l = repos.getIpList(app.db, req.params.id)
       if (!l) throw new AppError('NOT_FOUND', 'List not found', 404)
       const body = listEntriesBodySchema.parse(req.body)
+      const summary = listImportQuerySchema.parse(req.query).return === 'summary'
+      validateImportBudget(body.values ?? [], body.items?.length ?? 0)
       try {
-        const result = await addListEntries(app.db, l.id, {
+        const input = { values: body.values, items: body.items }
-          values: body.values,
+        const result = summary ? await addListEntriesSummary(app.db, l.id, input) : await addListEntries(app.db, l.id, input)
-          items: body.items,
-        })
         auditMutation(app, config, req, {
           action: 'list.entries.add',
           targetType: 'app_resource',
@@ -117,10 +152,10 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
           summary: `Добавлены записи в список: ${l.name}`,
           details: {
             list_id: l.id,
-            entry_count: result.entries.length,
+            entry_count: 'count' in result ? result.count : result.entries.length,
           },
         })
-        return mapListDetail(app.db, l.id) ?? result
+        return summary ? result : mapListDetail(app.db, l.id) ?? result
       } catch (err) {
         throw new AppError(
           'VALIDATION_ERROR',
@@ -137,8 +172,9 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
       const l = repos.getIpList(app.db, req.params.id)
       if (!l) throw new AppError('NOT_FOUND', 'List not found', 404)
       const body = deleteListEntryBodySchema.parse(req.body)
+      const summary = listImportQuerySchema.parse(req.query).return === 'summary'
       try {
-        await deleteListEntry(app.db, l.id, body.value)
+        const result = summary ? await deleteListEntrySummary(app.db, l.id, body.value) : await deleteListEntry(app.db, l.id, body.value)
         auditMutation(app, config, req, {
           action: 'list.entries.delete',
           severity: 'warning',
@@ -147,7 +183,7 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
           summary: `Удалена запись из списка: ${l.name}`,
           details: { list_id: l.id, value: body.value },
         })
-        return mapListDetail(app.db, l.id)
+        return summary ? result : mapListDetail(app.db, l.id)
       } catch (err) {
         throw new AppError(
           'VALIDATION_ERROR',
@@ -175,7 +211,9 @@ export const listsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
 
   app.delete<{ Params: { id: string } }>('/lists/:id', async (req) => {
     const l = repos.getIpList(app.db, req.params.id)
-    repos.deleteIpList(app.db, req.params.id)
+    try { repos.deleteIpList(app.db, req.params.id) } catch (err) {
+      throw new AppError('CONFLICT', err instanceof Error ? err.message : 'List is in use', 409)
+    }
     if (l) {
       auditMutation(app, config, req, {
         action: 'list.delete',

apps/api/src/routes/policy-rollouts.ts

@@ -0,0 +1,68 @@
+import type { FastifyPluginAsync } from 'fastify'
+import { repos } from '@evofw/db'
+import { createPolicySnapshotSchema, policySnapshotDiffSchema, policyPageSchema, createPolicyRolloutSchema, policyRolloutTransitionSchema, releasePolicyPinSchema } from '@evofw/shared'
+import type { AppConfig } from '../config.js'
+import { AppError } from '../plugins/error-handler.js'
+import { auditMutation } from '../services/audit.js'
+import { createPolicySnapshot, diffPolicySnapshot, startPolicyRollout, getPolicyRolloutView, confirmPolicyRollout, rollbackPolicyRollout, releaseAgentPolicyPin } from '../services/policy/rollouts.js'
+
+export const policyRolloutsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (app, { config }) => {
+  const audit = (req: Parameters<typeof auditMutation>[2], action: string, id: string) => auditMutation(app, config, req, {
+    action, targetType: 'app_resource', targetId: id, summary: action, details: { resource_id: id },
+  })
+  app.post('/policies/snapshots', async (req, reply) => {
+    const body = createPolicySnapshotSchema.parse(req.body)
+    const { payload_json: _, ...snapshot } = createPolicySnapshot(app.db, body.source_agent_id, body.name)
+    audit(req, 'policy_snapshot.create', snapshot.id)
+    return reply.code(201).send(snapshot)
+  })
+  app.get('/policies/snapshots', async (req) => {
+    const page = policyPageSchema.parse(req.query)
+    return { items: repos.listPolicySnapshots(app.db, page.limit, page.offset) }
+  })
+  app.get<{ Params: { id: string } }>('/policies/snapshots/:id', async (req) => {
+    const snapshot = repos.getPolicySnapshot(app.db, req.params.id)
+    if (!snapshot) throw new AppError('NOT_FOUND', 'Snapshot not found', 404)
+    const { payload_json, ...metadata } = snapshot
+    return { ...metadata, payload: JSON.parse(payload_json) }
+  })
+  app.delete<{ Params: { id: string } }>('/policies/snapshots/:id', async (req) => {
+    if (!repos.getPolicySnapshot(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Snapshot not found', 404)
+    if (!repos.deletePolicySnapshot(app.db, req.params.id)) throw new AppError('SNAPSHOT_IN_USE', 'Referenced snapshots cannot be deleted', 409)
+    audit(req, 'policy_snapshot.delete', req.params.id)
+    return { ok: true }
+  })
+  app.post<{ Params: { id: string } }>('/policies/snapshots/:id/diff', async (req) => diffPolicySnapshot(app.db, req.params.id, policySnapshotDiffSchema.parse(req.body).agent_ids))
+  app.post('/policies/rollouts', async (req, reply) => {
+    const rollout = startPolicyRollout(app.db, createPolicyRolloutSchema.parse(req.body))
+    audit(req, 'policy_rollout.start', rollout.id)
+    return reply.code(201).send(rollout)
+  })
+  app.get('/policies/rollouts', async (req) => {
+    const page = policyPageSchema.parse(req.query)
+    return { items: repos.listPolicyRollouts(app.db, page.limit, page.offset) }
+  })
+  app.get<{ Params: { id: string } }>('/policies/rollouts/:id', async (req) => getPolicyRolloutView(app.db, req.params.id))
+  app.delete<{ Params: { id: string } }>('/policies/rollouts/:id', async (req) => {
+    if (!repos.getPolicyRollout(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Rollout not found', 404)
+    if (!repos.deletePolicyRollout(app.db, req.params.id)) throw new AppError('ROLLOUT_IN_USE', 'Finish rollout and release every pin before deleting history', 409)
+    audit(req, 'policy_rollout.delete', req.params.id)
+    return { ok: true }
+  })
+  app.post<{ Params: { id: string } }>('/policies/rollouts/:id/confirm', async (req) => {
+    const result = confirmPolicyRollout(app.db, req.params.id, policyRolloutTransitionSchema.parse(req.body).expected_version)
+    audit(req, 'policy_rollout.confirm', result.id)
+    return result
+  })
+  app.post<{ Params: { id: string } }>('/policies/rollouts/:id/rollback', async (req) => {
+    const result = rollbackPolicyRollout(app.db, req.params.id, policyRolloutTransitionSchema.parse(req.body).expected_version)
+    audit(req, 'policy_rollout.rollback', result.id)
+    return result
+  })
+  app.post<{ Params: { id: string } }>('/agents/:id/policy-pin/release', async (req) => {
+    const body = releasePolicyPinSchema.parse(req.body)
+    const result = releaseAgentPolicyPin(app.db, req.params.id, body.rollout_id, body.expected_generation)
+    audit(req, 'policy_pin.release', req.params.id)
+    return result
+  })
+}

apps/api/src/routes/policy-sets.ts

@@ -59,7 +59,7 @@ export const policySetsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async
       description: body.description,
       enabled: body.enabled === undefined ? undefined : body.enabled ? 1 : 0,
     })
-    if (body.enabled !== undefined) {
+    if (body.enabled !== undefined || updated!.name !== s.name) {
       repos.bumpAgentsForSet(app.db, s.id)
     }
     auditMutation(app, config, req, {

apps/api/src/routes/settings.ts

@@ -3,6 +3,7 @@ import { repos } from '@evofw/db'
 import { putSettingsBodySchema } from '@evofw/shared'
 import type { AppConfig } from '../config.js'
 import { encryptSecret } from '../services/secret-cipher.js'
+import { auditMutation } from '../services/audit.js'
 
 export const settingsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   app,
@@ -10,30 +11,35 @@ export const settingsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
 ) => {
   const { config } = opts
 
-  app.get('/settings', async () => {
+  app.get('/settings', async (_req, reply) => {
+    reply.header('Cache-Control', 'no-store')
     const rows = repos.listSettings(app.db)
     const map: Record<string, string> = {}
     for (const r of rows) {
-      if (r.key === 'evobgp_api_token' && r.value) {
+      if ((r.key === 'evobgp_api_token' || r.key === 'enroll_seed') && r.value) {
         map[r.key] = '********'
       } else {
         map[r.key] = r.value
       }
     }
-    if (!map.enroll_seed) map.enroll_seed = config.enrollSeed
+    if (!map.enroll_seed && config.enrollSeed) map.enroll_seed = '********'
     return map
   })
 
   app.put('/settings', async (req) => {
     const body = putSettingsBodySchema.parse(req.body)
+    const changedKeys: string[] = []
     for (const [k, v] of Object.entries(body)) {
-      if (k === 'evobgp_api_token' && v === '********') continue
+      if ((k === 'evobgp_api_token' || k === 'enroll_seed') && v === '********') continue
+      if (repos.getSetting(app.db, k) === v && k !== 'evobgp_api_token') continue
       repos.setSetting(
         app.db,
         k,
         k === 'evobgp_api_token' ? encryptSecret(v) : v,
       )
+      changedKeys.push(k)
     }
+    if (changedKeys.length) auditMutation(app, config, req, { action: 'settings.update', targetType: 'app_resource', targetId: 'settings', summary: 'Обновлены настройки', details: { changed_keys: changedKeys } })
     return { ok: true }
   })
 }

apps/api/src/routes/stats.ts

@@ -4,6 +4,7 @@ import { AppError } from '../plugins/error-handler.js'
 import type { AppConfig } from '../config.js'
 import { auditMutation } from '../services/audit.js'
 import { mapAgent } from '../services/row-mappers.js'
+import { queryTimeseries, type TimeseriesQuery } from '../services/analytics.js'
 
 export const statsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
   app,
@@ -11,6 +12,47 @@ export const statsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
 ) => {
   const { config } = opts
 
+  app.get<{ Querystring: TimeseriesQuery }>('/stats/timeseries', async (req) => queryTimeseries(app.db, req.query))
+  app.get<{ Querystring: TimeseriesQuery }>('/stats/timeseries.csv', async (req, reply) => {
+    const result = queryTimeseries(app.db, req.query)
+    const rows = ['timestamp,packets_dropped,packets_accepted,pps,coverage,estimated,partial,truncated,lost_reports',
+      ...result.items.map((point) => [point.timestamp, point.packets_dropped ?? '', point.packets_accepted ?? '', point.pps ?? '', point.coverage, point.estimated, point.quality.partial, point.quality.truncated, point.quality.lost_reports].join(','))]
+    return reply.header('Content-Disposition', 'attachment; filename="evofw-timeseries.csv"').type('text/csv; charset=utf-8').send(rows.join('\r\n') + '\r\n')
+  })
+
+  app.get<{ Params: { id: string }; Querystring: { from?: string; to?: string; cursor?: string; limit?: string } }>('/agents/:id/stats/raw', async (req) => {
+    if (!repos.getAgent(app.db, req.params.id)) throw new AppError('NOT_FOUND', 'Agent not found', 404)
+    const from = req.query.from ?? new Date(Date.now() - 86_400_000).toISOString()
+    const to = req.query.to ?? new Date().toISOString()
+    const limit = req.query.limit === undefined ? 100 : Number(req.query.limit)
+    if (!Number.isFinite(Date.parse(from)) || !Number.isFinite(Date.parse(to)) || Date.parse(from) >= Date.parse(to) || !Number.isInteger(limit) || limit < 1 || limit > 500) throw new AppError('VALIDATION_ERROR', 'Invalid raw sample range or limit (1..500)', 400)
+    let cursor: { at: string; id: string } | undefined
+    if (req.query.cursor) {
+      try {
+        if (req.query.cursor.length > 512 || !/^[A-Za-z0-9_-]+$/.test(req.query.cursor)) throw new Error('Invalid cursor')
+        const decoded = JSON.parse(Buffer.from(req.query.cursor, 'base64url').toString('utf8')) as { at?: unknown; id?: unknown }
+        if (typeof decoded.at !== 'string' || !Number.isFinite(Date.parse(decoded.at)) || typeof decoded.id !== 'string' || decoded.id.length > 128) throw new Error('Invalid cursor')
+        cursor = { at: decoded.at, id: decoded.id }
+      } catch { throw new AppError('VALIDATION_ERROR', 'Invalid sample cursor', 400) }
+    }
+    const found = repos.listRawStatsPage(app.db, req.params.id, new Date(from).toISOString(), new Date(to).toISOString(), limit, cursor)
+    const hasMore = found.length > limit, selected = found.slice(0, limit), last = selected.at(-1)
+    return {
+      items: selected.map((sample) => ({
+        id: sample.id, agent_id: sample.agentId, recorded_at: sample.recordedAt, observed_at: sample.observedAt,
+        telemetry_version: sample.telemetryVersion, report_id: sample.reportId, sequence: sample.sequence,
+        packets_dropped: sample.droppedPresent === 0 ? null : sample.packetsDropped,
+        packets_accepted: sample.acceptedPresent === 0 ? null : sample.packetsAccepted,
+        delta_dropped: sample.deltaDropped, delta_accepted: sample.deltaAccepted,
+        dropped_interval_start: sample.droppedIntervalStart, accepted_interval_start: sample.acceptedIntervalStart,
+        interval_ms: sample.intervalMs, quality: sample.qualityJson ? JSON.parse(sample.qualityJson) : null,
+        detail_counts: sample.detailCountsJson ? JSON.parse(sample.detailCountsJson) : null,
+      })),
+      has_more: hasMore,
+      next_cursor: hasMore && last ? Buffer.from(JSON.stringify({ at: last.recordedAt, id: last.id })).toString('base64url') : null,
+    }
+  })
+
   app.get<{ Params: { id: string } }>('/agents/:id/stats', async (req) => ({
     items: repos.listStatsSamples(app.db, req.params.id).map((s) => ({
       id: s.id,
@@ -72,15 +114,8 @@ export const statsRoutes: FastifyPluginAsync<{ config: AppConfig }> = async (
     async (req) => {
       const agent = repos.getAgent(app.db, req.params.id)
       if (!agent) throw new AppError('NOT_FOUND', 'Agent not found', 404)
-      const updated = repos.updateAgent(app.db, agent.id, {
+      repos.resetDisplayedStats(app.db, agent.id)
-        lastApplyPacketsDropped: 0,
+      const updated = repos.getAgent(app.db, agent.id)
-        lastApplyPacketsAccepted: 0,
-        totalPacketsDropped: 0,
-        totalPacketsAccepted: 0,
-      })
-      repos.deleteStatsSamplesForAgent(app.db, agent.id)
-      repos.deleteIpBlockStatsForAgent(app.db, agent.id)
-      repos.deletePortBlockStatsForAgent(app.db, agent.id)
       auditMutation(app, config, req, {
         action: 'agent.stats_reset',
         severity: 'info',

apps/api/src/routes/ui-summary.ts

@@ -0,0 +1,32 @@
+import type { FastifyPluginAsync } from 'fastify'
+import { hasPermission, type UiSummary } from '@evofw/shared'
+
+/** Authentication comes from the protected API scope; each section is independent. */
+export const uiSummaryRoutes: FastifyPluginAsync = async (app) => {
+  app.get('/ui/summary', async (req) => {
+    const now = new Date()
+    const summary: UiSummary = { updated_at: now.toISOString() }
+    const permissions = req.authUser?.permissions ?? []
+    const allowed = (section: string) => req.authUser?.isAdmin || hasPermission(permissions, `fw:${section}:read`)
+    const agentsAllowed = allowed('agents'), dashboardAllowed = allowed('dashboard')
+    let counts: { total: number; approved: number; online: number; pending: number; dropped: number; accepted: number } | undefined
+    if (agentsAllowed || dashboardAllowed) {
+      counts = app.sqlite.prepare(`SELECT count(*) AS total,
+        coalesce(sum(status='approved'),0) AS approved,
+        coalesce(sum(status='approved' AND last_seen_at >= ?),0) AS online,
+        coalesce(sum(status='pending'),0) AS pending,
+        coalesce(sum(total_packets_dropped),0) AS dropped,
+        coalesce(sum(total_packets_accepted),0) AS accepted FROM agents`).get(new Date(now.getTime() - 5 * 60_000).toISOString()) as typeof counts
+    }
+    let lists: { total: number; failed: number } | undefined
+    if (allowed('lists') || dashboardAllowed) lists = app.sqlite.prepare("SELECT count(*) AS total, coalesce(sum(last_error IS NOT NULL AND last_error != ''),0) AS failed FROM ip_lists").get() as typeof lists
+    if (agentsAllowed && counts) summary.agents = { approved: counts.approved, online: counts.online, pending: counts.pending, offline: counts.approved - counts.online }
+    if (allowed('lists') && lists) summary.lists = lists
+    if (dashboardAllowed && counts && lists) summary.dashboard = {
+      agents_total: counts.total, agents_approved: counts.approved, agents_online: counts.online, agents_pending: counts.pending,
+      packets_dropped: counts.dropped, packets_accepted: counts.accepted, lists_total: lists.total,
+    }
+    if (allowed('policies')) summary.policies = app.sqlite.prepare('SELECT (SELECT count(*) FROM policy_sets) AS sets_total, (SELECT count(*) FROM policy_rules) AS rules_total').get() as UiSummary['policies']
+    return summary
+  })
+}

apps/api/src/routes/webhooks.ts

@@ -0,0 +1,28 @@
+import type { FastifyPluginAsync } from 'fastify'
+import { webhookBodySchema, webhookPatchSchema } from '@evofw/shared'
+import type { AppConfig } from '../config.js'
+import { Webhooks } from '../services/webhooks.js'
+import { auditMutation, recordAudit } from '../services/audit.js'
+
+export const webhooksRoutes: FastifyPluginAsync<{ config: AppConfig; memory?: boolean }> = async (app, { config, memory }) => {
+  const worker = new Webhooks(app.sqlite, config, undefined, () => {
+    app.log.warn('Webhook delivery capacity exhausted; notifications dropped')
+    recordAudit(app, config, { action: 'settings.update', targetType: 'app_resource', targetId: 'webhooks', summary: 'Webhook delivery capacity exhausted', details: { changed_keys: ['webhook_dropped_notifications'] } })
+  })
+  const audit = (request: Parameters<typeof auditMutation>[2], id: string, operation: string) => auditMutation(app, config, request, { action: 'settings.update', targetType: 'app_resource', targetId: id, summary: `Webhook ${operation}`, details: { changed_keys: ['webhooks'] } })
+  app.get('/webhooks', async () => worker.list())
+  app.post('/webhooks', async (request, reply) => {
+    const result = worker.create(webhookBodySchema.parse(request.body)); audit(request, result.id, 'created')
+    return reply.code(201).send(result)
+  })
+  app.patch<{ Params: { id: string } }>('/webhooks/:id', async request => {
+    const result = worker.update(request.params.id, webhookPatchSchema.parse(request.body)); audit(request, result.id, 'updated'); return result
+  })
+  app.delete<{ Params: { id: string } }>('/webhooks/:id', async request => {
+    const result = worker.remove(request.params.id); audit(request, request.params.id, 'deleted'); return result
+  })
+  app.get<{ Params: { id: string } }>('/webhooks/:id/deliveries', async request => worker.deliveries(request.params.id))
+  const timer = memory ? null : setInterval(() => { void worker.tick().catch(() => app.log.warn('Webhook worker failed')) }, 30_000)
+  timer?.unref()
+  app.addHook('onClose', async () => { if (timer) clearInterval(timer); await worker.close() })
+}

apps/api/src/services/agent-hardening-budget.test.ts

@@ -0,0 +1,64 @@
+import { spawnSync } from 'node:child_process'
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { afterEach, describe, expect, it } from 'vitest'
+const source = readFileSync(new URL('../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8')
+const python = process.platform === 'win32' ? 'python' : 'python3'
+const dirs: string[] = []
+function embedded(name: string) { const s = source.slice(source.indexOf(`${name}() {`)); const start = s.indexOf("<<'PY'") + 7; return s.slice(start, s.indexOf('\nPY', start)).trimStart() }
+function parse(patch: object) { const dir = mkdtempSync(join(tmpdir(), 'evofw-budget-')); dirs.push(dir); const file = join(dir, 'candidate.json'); writeFileSync(file, JSON.stringify({ hash: 'sha256:' + 'a'.repeat(64), generation: 1, default_action: 'accept', deny_cidrs: [], allow_cidrs: [], port_rules: [], ...patch })); const result = spawnSync(python, ['-c', embedded('parse_policy'), file, dir], { encoding: 'utf8' }); return { dir, result } }
+afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }) })
+function addresses(n: number) { return Array.from({ length: n }, (_, i) => { const v = i * 2; return `10.${(v >>> 16) & 255}.${(v >>> 8) & 255}.${v & 255}/32` }) }
+
+describe('agent effective policy budgets', () => {
+  it('accepts exactly 50k canonical unique memberships and rejects 50k+1 before emitting apply data', () => {
+    const accepted = parse({ deny_cidrs: addresses(50000) }); expect(accepted.result.status, accepted.result.stderr).toBe(0)
+    const rejected = parse({ deny_cidrs: addresses(50001) }); expect(rejected.result.status).not.toBe(0)
+    expect(readFileSync(join(rejected.dir, 'apply-error-code'), 'utf8')).toBe('POLICY_BUDGET_EXCEEDED')
+    expect(() => readFileSync(join(rejected.dir, 'normalized.json'))).toThrow()
+  }, 15000)
+  it('counts adjacent and overlapping unique CIDRs before optional kernel collapse', () => {
+    const adjacent = Array.from({ length: 50000 }, (_, i) => `10.${(i >>> 16) & 255}.${(i >>> 8) & 255}.${i & 255}/32`)
+    const accepted = parse({ deny_cidrs: adjacent }); expect(accepted.result.status, accepted.result.stderr).toBe(0)
+    expect(JSON.parse(readFileSync(join(accepted.dir, 'normalized.json'), 'utf8')).deny_cidrs.length).toBeLessThan(50000)
+    const rejected = parse({ deny_cidrs: [...adjacent, '10.0.0.0/8'] })
+    expect(rejected.result.status).not.toBe(0)
+    expect(readFileSync(join(rejected.dir, 'apply-error-code'), 'utf8')).toBe('POLICY_BUDGET_EXCEEDED')
+  }, 15000)
+  it('counts repeated CIDRs separately across distinct expanded Port ACL source sets', () => {
+    const src = addresses(1000)
+    const { dir, result } = parse({ port_rules: Array.from({ length: 51 }, (_, i) => ({ id: `rule-${i}`, action: 'open', protocol: 'tcp', port_start: 443, src_cidrs: src })) })
+    expect(result.status).not.toBe(0)
+    expect(readFileSync(join(dir, 'apply-error-code'), 'utf8')).toBe('POLICY_BUDGET_EXCEEDED')
+  })
+  it('caps raw arrays before expensive normalization and accepts large compressible raw input within the cap', () => {
+    const rejected = parse({ deny_cidrs: Array(200001).fill('192.0.2.1/32') })
+    expect(rejected.result.status).not.toBe(0)
+    expect(readFileSync(join(rejected.dir, 'apply-error-code'), 'utf8')).toBe('POLICY_BUDGET_EXCEEDED')
+    const accepted = parse({ deny_cidrs: Array(100000).fill('192.0.2.1/32') })
+    expect(accepted.result.status, accepted.result.stderr).toBe(0)
+    expect(JSON.parse(readFileSync(join(accepted.dir, 'normalized.json'), 'utf8')).deny_cidrs).toEqual(['192.0.2.1/32'])
+  }, 15000)
+  it('caps serialized Port ACL rule count', () => {
+    const { dir, result } = parse({ port_rules: Array.from({ length: 1001 }, (_, i) => ({ id: `rule-${i}`, action: 'close', protocol: 'tcp', port_start: 22, src_cidrs: [] })) })
+    expect(result.status).not.toBe(0)
+    expect(readFileSync(join(dir, 'apply-error-code'), 'utf8')).toBe('POLICY_BUDGET_EXCEEDED')
+  })
+  it('rejects an oversized JSON document before parsing it', () => {
+    const { dir } = parse({})
+    writeFileSync(join(dir, 'oversized.json'), ' '.repeat(8388609))
+    const result = spawnSync(python, ['-c', embedded('parse_policy'), join(dir, 'oversized.json'), dir], { encoding: 'utf8' })
+    expect(result.status).not.toBe(0)
+    expect(readFileSync(join(dir, 'apply-error-code'), 'utf8')).toBe('POLICY_BUDGET_EXCEEDED')
+  })
+  it('accepts protocol=both expansion through tuple uniqueness and emits distinct stable traffic identifiers', () => {
+    const { dir, result } = parse({ port_rules: ['tcp', 'udp'].map((protocol) => ({ id: 'shared-id', action: 'open', protocol, port_start: 443, src_cidrs: ['0.0.0.0/0', '::/0'] })) })
+    expect(result.status, result.stderr).toBe(0)
+    const generated = spawnSync(python, ['-c', embedded('render_nft_batch'), join(dir, 'normalized.json'), '1', '1', '0'], { encoding: 'utf8' })
+    expect(generated.status, generated.stderr).toBe(0)
+    const comments = [...generated.stdout.matchAll(/input .*comment "(evofw-port-[a-f0-9]+-(?:tcp|udp)-v[46])"/g)].map((match) => match[1])
+    expect(comments).toHaveLength(4)
+    expect(new Set(comments).size).toBe(4)
+  })
+})

apps/api/src/services/agent-hardening-cache.test.ts

@@ -0,0 +1,63 @@
+import { spawnSync } from 'node:child_process'
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { afterEach, describe, expect, it } from 'vitest'
+const source = readFileSync(new URL('../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8')
+const python = process.platform === 'win32' ? 'python' : 'python3'
+const bash = process.platform === 'win32' ? 'C:/Program Files/Git/bin/bash.exe' : 'bash'
+const pythonPath = spawnSync(python, ['-c', 'import sys;print(sys.executable)'], { encoding: 'utf8' }).stdout.trim().replaceAll('\\', '/')
+const dirs: string[] = []
+function workspace() { const d = mkdtempSync(join(tmpdir(), 'evofw-cache-')); dirs.push(d); return d }
+afterEach(() => { for (const d of dirs.splice(0)) rmSync(d, { recursive: true, force: true }) })
+function embedded(name: string) { const s = source.slice(source.indexOf(`${name}() {`)); const a = s.indexOf("<<'PY'") + 7; return s.slice(a, s.indexOf('\nPY', a)).trimStart() }
+function worker(d: string, op: string, ...args: string[]) { return spawnSync(python, ['-c', embedded('telemetry_worker'), d, d, op, ...args], { encoding: 'utf8', env: { ...process.env, EVOFW_TEST_BOOT_ID: 'cache-boot' } }) }
+const policy = (letter = 'a', generation = 1) => ({ hash: 'sha256:' + letter.repeat(64), generation, default_action: 'accept', deny_cidrs: [], allow_cidrs: [], port_rules: [] })
+function cache(d: string) { const body = JSON.stringify(policy()); writeFileSync(join(d, 'candidate.json'), body); writeFileSync(join(d, 'normalized.json'), body); writeFileSync(join(d, 'policy-http-status'), '200'); writeFileSync(join(d, 'policy-headers.txt'), 'HTTP/1.1 200 OK\r\nETag: "etag-first"\r\n'); expect(worker(d, 'init').status).toBe(0); expect(worker(d, 'commit-policy').status).toBe(0); writeFileSync(join(d, 'last_hash'), policy().hash); writeFileSync(join(d, 'kernel_state'), 'nft old-digest\n') }
+const workerFunctions = source.slice(source.indexOf('telemetry_worker() {'), source.indexOf('telemetry_worker init'))
+const requestFunction = source.slice(source.indexOf('curl_policy() {'), source.indexOf('# Do not use `if ! cmd;'))
+const parseFunction = source.slice(source.indexOf('parse_policy() {'), source.indexOf('HASH="";'))
+function shell(d: string, content: string) { writeFileSync(join(d, 'test.sh'), `set -euo pipefail\nSTATE_DIR=.; RUN_DIR=.; HASH_FILE=last_hash; POLICY_FILE=last_policy.json; EVOFW_CP_URL=https://unused.invalid; CLIENT_TOKEN=fixture; BACKEND=nft; DENY=(); ALLOW=(); APPLY_ERROR_CODE=apply_failed;\npython3() { '${pythonPath}' "$@"; }\nlog() { :; }\n${workerFunctions}\n${requestFunction}\n${parseFunction}\n${content}`); return spawnSync(bash, ['--noprofile', '--norc', join(d, 'test.sh').replaceAll('\\', '/')], { cwd: d, encoding: 'utf8', timeout: 15000 }) }
+function curl(status: number) { return `curl() { printf '%s\\n' "$*" > curl-args; local dest= headers=; while (($#)); do case "$1" in -o) dest=$2; shift 2;; -D) headers=$2; shift 2;; *) shift;; esac; done; printf 'HTTP/1.1 ${status} result\\r\\nETag: "etag-next"\\r\\n' > "$headers"; ${status === 304 ? ': > "$dest"' : 'cp response.json "$dest"'}; printf '${status}'; }\n` }
+
+describe('agent last-good policy validator cache and typed upstream rejections', () => {
+  it('makes a cold-cache request without a validator', () => {
+    const d = workspace(); expect(worker(d, 'init').status).toBe(0); writeFileSync(join(d, 'response.json'), JSON.stringify(policy()))
+    const r = shell(d, curl(200) + 'curl_policy candidate.json\nparse_policy candidate.json\n')
+    expect(r.status, r.stderr).toBe(0); expect(readFileSync(join(d, 'curl-args'), 'utf8')).not.toContain('If-None-Match')
+    expect(() => readFileSync(join(d, 'last_policy.json'))).toThrow()
+  })
+  it('reuses a matching durable body on304 and still repairs drift instead of skipping sync', () => {
+    const d = workspace(); cache(d)
+    const main = source.slice(source.indexOf('KERNEL_STATE_FILE="$STATE_DIR/kernel_state"'))
+    const r = shell(d, curl(304) + `type() { return 0; }\nkernel_digest() { echo repaired; }\ncollect_telemetry() { :; }\nflush_reports() { :; }\nsend_report() { :; }\napply_nft() { echo apply >> applied; KERNEL_METHOD=nft; APPLIED=0; APPLIED_FAMILIES=4,6; }\ncurl_policy candidate.json\nparse_policy candidate.json\n${main}`)
+    expect(r.status, r.stderr).toBe(0)
+    expect(readFileSync(join(d, 'curl-args'), 'utf8')).toContain('If-None-Match: "etag-first"')
+    expect(readFileSync(join(d, 'applied'), 'utf8').trim()).toBe('apply')
+    const saved = JSON.parse(readFileSync(join(d, 'last_policy.json'), 'utf8'))
+    expect(saved.hash).toBe(policy().hash); expect(saved._response_etag).toBe('"etag-first"'); expect(JSON.parse(saved._response_body)).toEqual(policy())
+  })
+  it('preserves the durable matching body and validator when a new candidate fails kernel apply', () => {
+    const d = workspace(); cache(d); const before = readFileSync(join(d, 'last_policy.json'), 'utf8')
+    writeFileSync(join(d, 'response.json'), JSON.stringify(policy('b', 2)))
+    const main = source.slice(source.indexOf('KERNEL_STATE_FILE="$STATE_DIR/kernel_state"'))
+    const r = shell(d, curl(200) + `type() { return 0; }\nkernel_digest() { echo repaired; }\ncollect_telemetry() { :; }\nflush_reports() { :; }\napply_nft() { return 1; }\ncurl_policy candidate.json\nparse_policy candidate.json\n${main}`)
+    expect(r.status).not.toBe(0); expect(readFileSync(join(d, 'last_policy.json'), 'utf8')).toBe(before)
+  })
+  it('does not accept304 without a durable validated body', () => {
+    const d = workspace(); expect(worker(d, 'init').status).toBe(0)
+    const r = shell(d, curl(304) + 'if curl_policy candidate.json; then exit 9; fi\n')
+    expect(r.status, r.stderr).toBe(0); expect(() => readFileSync(join(d, 'last_policy.json'))).toThrow()
+  })
+  it.each([
+    [422, 'POLICY_BUDGET_EXCEEDED', 1, 'POLICY_BUDGET_EXCEEDED'],
+    [422, 'SOME_OTHER_ERROR', 1, 'apply_failed'],
+    [403, 'POLICY_BUDGET_EXCEEDED', 2, 'apply_failed'],
+  ])('handles HTTP%s error %s without false policy acknowledgement', (status, code, expectedRc, expectedCode) => {
+    const d = workspace(); cache(d); const before = readFileSync(join(d, 'last_policy.json'), 'utf8')
+    writeFileSync(join(d, 'response.json'), JSON.stringify({ error: { code } }))
+    const r = shell(d, curl(status as number) + 'rc=0; curl_policy candidate.json || rc=$?; printf "%s %s" "$rc" "$APPLY_ERROR_CODE" > result\n')
+    expect(r.status, r.stderr).toBe(0); expect(readFileSync(join(d, 'result'), 'utf8')).toBe(`${expectedRc} ${expectedCode}`)
+    expect(readFileSync(join(d, 'last_policy.json'), 'utf8')).toBe(before)
+  })
+})

apps/api/src/services/agent-hardening-credentials.test.ts

@@ -0,0 +1,148 @@
+import { readFileSync, mkdtempSync, writeFileSync, rmSync, existsSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { spawnSync } from 'node:child_process'
+import { describe, expect, it } from 'vitest'
+const linux = readFileSync(new URL('../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8').replace(/\r\n/g, '\n')
+const install = readFileSync(new URL('../agent-scripts/install.sh', import.meta.url), 'utf8').replace(/\r\n/g, '\n')
+const router = readFileSync(new URL('../agent-scripts/mikrotik-install.rsc', import.meta.url), 'utf8').replace(/\r\n/g, '\n')
+const worker = linux.split("<<'PY_CREDENTIALS'\n")[1]!.split('\nPY_CREDENTIALS')[0]!
+const fakeTransport = String.raw`
+import json, os, sys, urllib.request, urllib.error
+from pathlib import Path
+server_path=Path(sys.argv[1])/'server.json'
+# Windows test harness only: real Linux directory fsync is retained in production.
+real_open=os.open
+def test_open(path, flags, *args):
+    if os.path.isdir(path): return real_open(str(Path(sys.argv[1])/'directory-fsync-fixture'), os.O_RDWR|os.O_CREAT, 0o600)
+    return real_open(path,flags,*args)
+os.open=test_open
+class Response:
+    def __init__(self,value): self.value=value
+    def __enter__(self): return self
+    def __exit__(self,*args): pass
+    def read(self,*args): return json.dumps(self.value).encode()
+def fetch(req, timeout):
+    server=json.loads(server_path.read_text())
+    auth=req.headers['Authorization'][7:]
+    if req.full_url.endswith('/credentials'):
+        if auth!=server['current']: raise urllib.error.HTTPError(req.full_url,401,'invalid',{},None)
+        return Response({'client_id':'agent-test','rotation':None if server.get('next') or server.get('expired') else {'request_id':server['request_id']},'completed_rotation':{'request_id':server['request_id']} if server.get('next') else None})
+    if server.get('expired') or server.get('reject_exchange'): raise urllib.error.HTTPError(req.full_url,401,'expired',{},None)
+    body=json.loads(req.data)
+    if body['request_id']!=server['request_id']: raise urllib.error.URLError('request mismatch')
+    if server.get('next') and body['next_token']!=server['next']: raise urllib.error.URLError('replay mismatch')
+    if auth not in [server['current'],server.get('previous')]: raise urllib.error.HTTPError(req.full_url,401,'invalid',{},None)
+    server['previous']=auth; server['current']=body['next_token']; server['next']=body['next_token']
+    lose=server.pop('lose_ack',False)
+    server_path.write_text(json.dumps(server))
+    if lose: raise urllib.error.URLError('ACK lost after committed exchange')
+    return Response({'ok':True,'request_id':body['request_id'],'client_id':'wrong' if server.get('bad_ack') else 'agent-test'})
+urllib.request.urlopen=fetch
+`
+function fixture(extra: object = {}) {
+  const dir = mkdtempSync(join(tmpdir(), 'evofw-credential-'))
+  const old = 'evofw_' + 'a'.repeat(43)
+  const conf = join(dir, 'agent.conf')
+  writeFileSync(conf, `CLIENT_TOKEN='${old}'\nCLIENT_ID='agent-test'\n`)
+  writeFileSync(join(dir, 'server.json'), JSON.stringify({ current: old, request_id: '12345678-1234-4234-8234-123456789abc', ...extra }))
+  const run = () => spawnSync('python', ['-c', fakeTransport + '\n' + worker, dir, conf, 'https://fw.example.com', old, 'agent-test'], { encoding: 'utf8' })
+  return { dir, conf, old, run }
+}
+describe('credential client restart and ACK recovery (mock transport; no native firewall)', () => {
+  it('retains exactly one new token after lost ACK and resumes it on restart before committing active config', () => {
+    const f = fixture({ lose_ack: true })
+    try {
+      const first = f.run()
+      expect(first.status, first.stderr).toBe(1)
+      const pending = JSON.parse(readFileSync(join(f.dir, 'token-rotation-pending.json'), 'utf8').replace(/\r\n/g, '\n'))
+      expect(readFileSync(f.conf, 'utf8').replace(/\r\n/g, '\n')).toContain(f.old)
+      const next = pending.next_token
+      const retry = f.run()
+      expect(retry.status, retry.stderr).toBe(0)
+      expect(retry.stdout.trim()).toBe(next)
+      expect(readFileSync(f.conf, 'utf8').replace(/\r\n/g, '\n')).toContain(next)
+      expect(readFileSync(f.conf, 'utf8').replace(/\r\n/g, '\n')).not.toContain(f.old)
+      expect(existsSync(join(f.dir, 'token-rotation-pending.json'))).toBe(false)
+      expect(first.stderr + retry.stderr).not.toContain(next)
+    } finally { rmSync(f.dir, { recursive: true, force: true }) }
+  })
+  it('recovers a committed candidate after old-token replay expires during offline restart', () => {
+    const f = fixture({ lose_ack: true })
+    try {
+      expect(f.run().status).toBe(1)
+      const pending = JSON.parse(readFileSync(join(f.dir, 'token-rotation-pending.json'), 'utf8'))
+      const server = JSON.parse(readFileSync(join(f.dir, 'server.json'), 'utf8'))
+      writeFileSync(join(f.dir, 'server.json'), JSON.stringify({ ...server, expired: true }))
+      const resumed = f.run()
+      expect(resumed.status, resumed.stderr).toBe(0)
+      expect(readFileSync(f.conf, 'utf8')).toContain(pending.next_token)
+      expect(existsSync(join(f.dir, 'token-rotation-pending.json'))).toBe(false)
+    } finally { rmSync(f.dir, { recursive: true, force: true }) }
+  })
+  it('abandons an uncommitted expired exchange only after old-token proof, and reconciles a later request', () => {
+    const f = fixture({ expired: true })
+    try {
+      const requestId = '12345678-1234-4234-8234-123456789abc'
+      writeFileSync(join(f.dir, 'token-rotation-pending.json'), JSON.stringify({ request_id: requestId, old_token: f.old, next_token: 'evofw_' + 'b'.repeat(43), origin: 'https://fw.example.com', client_id: 'agent-test' }))
+      const resumed = f.run()
+      expect(resumed.status, resumed.stderr).toBe(0) // Caller continues policy and telemetry with this credential.
+      expect(resumed.stdout.trim()).toBe(f.old)
+      expect(readFileSync(f.conf, 'utf8')).toContain(f.old)
+      expect(existsSync(join(f.dir, 'token-rotation-pending.json'))).toBe(false)
+      const server = JSON.parse(readFileSync(join(f.dir, 'server.json'), 'utf8'))
+      expect(server.current).toBe(f.old)
+      writeFileSync(join(f.dir, 'server.json'), JSON.stringify({ ...server, expired: false, request_id: '22345678-1234-4234-8234-123456789abc' }))
+      const newer = f.run()
+      expect(newer.status, newer.stderr).toBe(0)
+      expect(newer.stdout.trim()).not.toBe(f.old)
+    } finally { rmSync(f.dir, { recursive: true, force: true }) }
+  })
+  it('retains pending state when the same request is still active after an exchange refusal', () => {
+    const f = fixture({ reject_exchange: true })
+    try {
+      expect(f.run().status).toBe(1)
+      const bytes = readFileSync(join(f.dir, 'token-rotation-pending.json'), 'utf8')
+      expect(f.run().status).toBe(1)
+      expect(readFileSync(join(f.dir, 'token-rotation-pending.json'), 'utf8')).toBe(bytes)
+      expect(readFileSync(f.conf, 'utf8')).toContain(f.old)
+    } finally { rmSync(f.dir, { recursive: true, force: true }) }
+  })
+  it('keeps the active token and pending exchange unchanged after a mismatched ACK', () => {
+    const f = fixture({ bad_ack: true })
+    try {
+      expect(f.run().status).toBe(1)
+      const bytes = readFileSync(join(f.dir, 'token-rotation-pending.json'), 'utf8').replace(/\r\n/g, '\n')
+      expect(f.run().status).toBe(1)
+      expect(readFileSync(join(f.dir, 'token-rotation-pending.json'), 'utf8').replace(/\r\n/g, '\n')).toBe(bytes)
+      expect(readFileSync(f.conf, 'utf8').replace(/\r\n/g, '\n')).toContain(f.old)
+    } finally { rmSync(f.dir, { recursive: true, force: true }) }
+  })
+  it('persists scoped fresh-enrollment state before the POST and active config before cleanup', () => {
+    expect(install).not.toContain('X-EvoFW-Seed:')
+    expect(install).toContain('X-EvoFW-Enrollment-Credential:')
+    expect(install.indexOf("<<'PY_ENROLL_PENDING'")).toBeLessThan(install.indexOf('ENROLL_CODE='))
+    expect(install.indexOf('write_conf "$CLIENT_ID" "$CLIENT_TOKEN" "$EVOFW_CLIENT_NAME"')).toBeLessThan(install.indexOf('rm -f "$ENROLL_PENDING"'))
+    expect(install).toContain('0o600')
+    expect(install).toContain('os.fsync')
+    expect(linux.indexOf('ACTIVE_TOKEN=$(credential_sync)')).toBeLessThan(linux.indexOf('\nmaybe_self_update\n'))
+    expect(linux.indexOf('CLIENT_TOKEN="$ACTIVE_TOKEN"')).toBeLessThan(linux.indexOf('\nmaybe_self_update\n'))
+  })
+  it('RouterOS saves pending enrollment/rotation before POST and uses the persisted active token after restart', () => {
+    expect(router).not.toContain('X-EvoFW-Seed:')
+    expect(router).not.toContain('EvofwSeed')
+    const enroll = router.indexOf('/v1/agent/enroll')
+    expect(router.indexOf('contents=[:serialize $credentials', router.indexOf('"enrollment_token"'))).toBeLessThan(enroll)
+    const exchange = router.indexOf('/v1/agent/token-rotation')
+    const pending = router.indexOf(':set ($credentials->"rotation") {')
+    expect(router.indexOf('contents=[:serialize $credentials', pending)).toBeLessThan(exchange)
+    const active = router.indexOf(':set ($credentials->"active_token") ($rotation->"next_token")')
+    expect(active).toBeGreaterThan(router.indexOf('credential ACK mismatch'))
+    expect(router.indexOf('contents=[:serialize $credentials', active)).toBeLessThan(router.indexOf(':set EvofwToken ($credentials->"active_token")', active))
+    expect(router).toContain('exchange request still active')
+    expect(router).toContain(':local abandoned false')
+    expect(router).toContain(':if (!$abandoned) do={')
+    expect(router).toContain('Continue policy/telemetry using the proven old credential')
+    expect(router).toContain('flash/evofw-credentials.json')
+  })
+})

apps/api/src/services/agent-hardening-dualstack.test.ts

@@ -0,0 +1,123 @@
+import { spawnSync } from 'node:child_process'
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { afterEach, describe, expect, it } from 'vitest'
+import { applyReportV2BodySchema } from '@evofw/shared'
+const script = readFileSync(new URL('../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8')
+const python = process.platform === 'win32' ? 'python' : 'python3'
+const bash = process.platform === 'win32' ? 'C:/Program Files/Git/bin/bash.exe' : 'bash'
+const py = spawnSync(python, ['-c', 'import sys;print(sys.executable)'], { encoding: 'utf8' }).stdout.trim().replaceAll('\\', '/')
+const dirs: string[] = []
+function dir() { const p = mkdtempSync(join(tmpdir(), 'evofw-dual-')); dirs.push(p); return p }
+afterEach(() => { for (const d of dirs.splice(0)) rmSync(d, { recursive: true, force: true }) })
+function embedded(name: string) { const s = script.slice(script.indexOf(`${name}() {`)); const a = s.indexOf("<<'PY'") + 7; return s.slice(a, s.indexOf('\nPY', a)).trimStart() }
+function runPython(code: string, args: string[]) { return spawnSync(python, ['-c', code, ...args], { encoding: 'utf8', env: { ...process.env, EVOFW_TEST_BOOT_ID: 'dual-boot' } }) }
+function normalized(patch = {}) { const d = dir(); const p = { hash: 'sha256:' + 'a'.repeat(64), generation: 1, default_action: 'accept', deny_cidrs: ['192.0.2.1', '2001:0db8:1::9/64'], allow_cidrs: ['2001:db8:2::/64'], port_rules: [], ...patch }; writeFileSync(join(d, 'candidate.json'), JSON.stringify(p)); const r = runPython(embedded('parse_policy'), [join(d, 'candidate.json'), d]); expect(r.status, r.stderr).toBe(0); return d }
+function shell(content: string) { const d = dir(); writeFileSync(join(d, 'ipv6_required'), '1'); writeFileSync(join(d, 'test.sh'), `set -euo pipefail\nRUN_DIR=.; python3() { '${py}' "$@"; }\n${content}`); const r = spawnSync(bash, ['--noprofile', '--norc', join(d, 'test.sh').replaceAll('\\', '/')], { cwd: d, encoding: 'utf8' }); return { d, r } }
+const ipsetFunctions = script.slice(script.indexOf('prepare_ipset_rules() {'), script.indexOf('collect_telemetry() {'))
+
+describe('agent hardening Linux dual-stack enforcement and rollback', () => {
+  it('canonicalizes and collapses each IP family independently', () => {
+    const d = normalized({ deny_cidrs: ['10.0.0.1/25', '10.0.0.128/25', '2001:0db8::1/65', '2001:db8:0:0:8000::/65'] })
+    const p = JSON.parse(readFileSync(join(d, 'normalized.json'), 'utf8'))
+    expect(p.deny_cidrs).toEqual(['10.0.0.0/24', '2001:db8::/64'])
+    expect(readFileSync(join(d, 'ipv6_required'), 'utf8')).toBe('1')
+  })
+  it('emits both family sets, deny/allow enforcement, and family-scoped any-source ACL without IPv4-any IPv6 bypass', () => {
+    const d = normalized({ port_rules: [
+      { id: 'v4-any', action: 'open', protocol: 'tcp', port_start: 443, src_cidrs: ['0.0.0.0/0'] },
+      { id: 'v6-corp', action: 'open', protocol: 'tcp', port_start: 443, src_cidrs: ['2001:db8:2::/64'] },
+    ] })
+    const result = runPython(embedded('render_nft_batch'), [join(d, 'normalized.json'), '1', '1', '1'])
+    expect(result.status, result.stderr).toBe(0)
+    expect(result.stdout).toContain('deny_v6 { type ipv6_addr; flags interval; counter; }')
+    expect(result.stdout).toContain('add element inet evofw deny_v6 { 2001:db8:1::/64 }')
+    expect(result.stdout).toContain('ip6 saddr @deny_v6 counter drop')
+    expect(result.stdout).toContain('ip6 saddr @allow_v6 counter accept')
+    expect(result.stdout).toContain('meta nfproto ipv4 tcp dport 443 counter accept')
+    expect(result.stdout).not.toContain('prerouting fib daddr type local tcp dport 443 counter accept')
+    expect(result.stdout).toContain('forward ct status dnat ip6 saddr @port_src_1_v6 tcp dport 443 counter accept')
+    expect(result.stdout).toContain('deny_port_hits_v6 { type ipv6_addr . inet_proto . inet_service;')
+    expect(result.stdout).toContain('input tcp dport 443 counter drop comment "evofw-port-implicit-tcp-443"')
+  })
+  it('restores the already-applied IPv4 rules and all four sets when IPv6 rule commit fails', () => {
+    const { d, r } = shell(`PORT_RULES_COUNT=0; DEFAULT_ACTION=drop; DENY=(192.0.2.1/32 2001:db8::1/128); ALLOW=();\nlog() { :; }\ntype() { return 0; }\nipset() { echo "$*" >> sets; if [[ "$1" == save ]]; then if [[ "$2" == *v6 ]]; then echo "create $2 hash:net family inet6 counters"; else echo "create $2 hash:net family inet counters"; fi; elif [[ "$1" == restore ]]; then cat >sets.restore; fi; }\niptables-save() { printf '%s\\n' ':EVOFW_INPUT - [0:0]' '-A INPUT -j ACCEPT' '-A INPUT -j EVOFW_INPUT' '-A EVOFW_INPUT -j REJECT'; }\nip6tables-save() { iptables-save; }\niptables-restore() { body=$(cat); if [[ "$*" == *--test* ]]; then echo check4 >> calls; elif [[ "$body" == *'EVOFW_INPUT -j REJECT'* ]]; then echo rollback4 >> calls; else echo commit4 >> calls; fi; }\nip6tables-restore() { cat >/dev/null; if [[ "$*" == *--test* ]]; then echo check6 >> calls; else echo fail6 >> calls; return 1; fi; }\n${ipsetFunctions}\nif apply_ipset; then exit 9; fi\n`)
+    expect(r.status, r.stderr).toBe(0)
+    expect(readFileSync(join(d, 'calls'), 'utf8').trim().split(/\r?\n/)).toEqual(['check4', 'check6', 'commit4', 'fail6', 'rollback4'])
+    const sets = readFileSync(join(d, 'sets'), 'utf8')
+    for (const family of [4, 6]) for (const kind of ['deny', 'allow']) expect(sets.match(new RegExp(`swap evofw_${kind}_stage_v${family} evofw_${kind}_v${family}`, 'g'))).toHaveLength(2)
+    const restored = readFileSync(join(d, 'sets.restore'), 'utf8')
+    expect(restored).toContain('add evofw_deny_stage_v4 192.0.2.1/32')
+    expect(restored).toContain('add evofw_deny_stage_v6 2001:db8::1/128')
+  })
+  it('refuses unavailable IPv6 capability before touching either active family', () => {
+    const { d, r } = shell(`PORT_RULES_COUNT=0; DEFAULT_ACTION=accept; DENY=(2001:db8::1/128); ALLOW=();\nlog() { :; }\ntype() { return 1; }\nipset() { echo mutation >> mutations; }\n${ipsetFunctions}\nif apply_ipset; then exit 9; fi\nprintf '%s' "$APPLY_ERROR_CODE" >code\n`)
+    expect(r.status, r.stderr).toBe(0)
+    expect(readFileSync(join(d, 'code'), 'utf8')).toBe('UNSUPPORTED_CAPABILITY')
+    expect(() => readFileSync(join(d, 'mutations'))).toThrow()
+  })
+  it('rejects an IPv6 owned jump preempted by a foreign rule during drift verification', () => {
+    const d = dir(); writeFileSync(join(d, 'live-ipset.txt'), 'create evofw_deny_v4 hash:net family inet\ncreate evofw_deny_v6 hash:net family inet6\n')
+    writeFileSync(join(d, 'live-iptables.txt'), '-A INPUT -j EVOFW_INPUT\n')
+    writeFileSync(join(d, 'live-iptables-v6.txt'), '-A INPUT -j ACCEPT\n-A INPUT -j EVOFW_INPUT\n')
+    const r = runPython(embedded('kernel_digest'), ['ipset', d])
+    expect(r.status).not.toBe(0); expect(r.stderr).toContain('owned INPUT jump is missing or preempted')
+  })
+  it('collects canonical IPv6 host/port counters and advertises only observed implemented capabilities', () => {
+    const d = dir(); expect(runPython(embedded('telemetry_worker'), [d, d, 'init']).status).toBe(0)
+    writeFileSync(join(d, 'metrics-nft.json'), JSON.stringify({ nftables: [
+      { set: { family: 'inet', table: 'evofw', name: 'allow_v6', elem: [] } },
+      { set: { family: 'inet', table: 'evofw', name: 'deny_v6', elem: [{ elem: { val: { prefix: { addr: '2001:0db8::1', len: 128 } }, counter: { packets: 9 } } }] } },
+      { set: { family: 'inet', table: 'evofw', name: 'deny_port_hits_v6', elem: [{ elem: { val: { concat: ['2001:0db8::1', 'tcp', 443] }, counter: { packets: 8 } } }] } },
+      { rule: { family: 'inet', table: 'evofw', chain: 'input', comment: 'evofw-v2/input/deny6', expr: [{ counter: { packets: 9 } }, { drop: null }] } },
+    ] }))
+    expect(runPython(embedded('telemetry_worker'), [d, d, 'capture', 'nft']).status).toBe(0)
+    expect(runPython(embedded('telemetry_worker'), [d, d, 'enqueue', 'ok', 'nft']).status).toBe(0)
+    const state = JSON.parse(readFileSync(join(d, 'telemetry-state.json'), 'utf8')); const report = JSON.parse(state.pending[0])
+    expect(report.ip_hits).toEqual([{ ip: '2001:db8::1', packets: 9 }])
+    expect(report.port_hits).toEqual([{ ip: '2001:db8::1', protocol: 'tcp', port: 443, packets: 8 }])
+    expect(report.capabilities).toEqual(expect.arrayContaining(['policy-ipv6', 'policy-port-acl-ipv6', 'ip-counters-ipv6', 'port-counters-ipv6']))
+    expect(applyReportV2BodySchema.safeParse(report).success).toBe(true)
+  })
+  it('omits aggregate totals when the active IPv6 family read fails, then recovers both-family totals', () => {
+    const d = dir(); expect(runPython(embedded('telemetry_worker'), [d, d, 'init']).status).toBe(0)
+    expect(runPython(embedded('telemetry_worker'), [d, d, 'rotate', '4,6']).status).toBe(0)
+    writeFileSync(join(d, 'metrics-iptables.txt'), '[5:100] -A EVOFW_INPUT -m set --match-set evofw_deny_v4 src -j DROP\n[2:100] -A EVOFW_INPUT -m set --match-set evofw_allow_v4 src -j ACCEPT\n')
+    expect(runPython(embedded('telemetry_worker'), [d, d, 'capture', 'ipset']).status).toBe(0)
+    const missing = JSON.parse(readFileSync(join(d, 'snapshot.json'), 'utf8'))
+    expect(missing).not.toHaveProperty('packets_dropped')
+    expect(missing).not.toHaveProperty('packets_accepted')
+    expect(missing.quality.partial).toBe(true)
+    expect(missing.quality.unsupported).toContain('ip6tables_counters_unavailable')
+    writeFileSync(join(d, 'metrics-iptables-v6.txt'), '[7:100] -A EVOFW_INPUT -m set --match-set evofw_deny_v6 src -j DROP\n[3:100] -A EVOFW_INPUT -m set --match-set evofw_allow_v6 src -j ACCEPT\n')
+    expect(runPython(embedded('telemetry_worker'), [d, d, 'capture', 'ipset']).status).toBe(0)
+    const recovered = JSON.parse(readFileSync(join(d, 'snapshot.json'), 'utf8'))
+    expect(recovered.packets_dropped).toBe(12)
+    expect(recovered.packets_accepted).toBe(5)
+    expect(recovered.quality.unsupported).not.toContain('ip6tables_counters_unavailable')
+  })
+
+  it('preserves complete part baselines across missing-family reports and catches reset masked by other-family growth', () => {
+    const d = dir(); expect(runPython(embedded('telemetry_worker'), [d, d, 'init']).status).toBe(0)
+    expect(runPython(embedded('telemetry_worker'), [d, d, 'rotate', '4,6']).status).toBe(0)
+    const v4 = (n: number) => writeFileSync(join(d, 'metrics-iptables.txt'), `[${n}:100] -A EVOFW_INPUT -m set --match-set evofw_deny_v4 src -j DROP\n`)
+    const v6 = (n: number) => writeFileSync(join(d, 'metrics-iptables-v6.txt'), `[${n}:100] -A EVOFW_INPUT -m set --match-set evofw_deny_v6 src -j DROP\n`)
+    const emit = () => {
+      expect(runPython(embedded('telemetry_worker'), [d, d, 'capture', 'ipset']).status).toBe(0)
+      expect(runPython(embedded('telemetry_worker'), [d, d, 'enqueue', 'ok', 'ipset']).status).toBe(0)
+      return JSON.parse(readFileSync(join(d, 'telemetry-state.json'), 'utf8'))
+    }
+    v4(5); v6(7); const before = emit()
+    v4(20); rmSync(join(d, 'metrics-iptables-v6.txt')); const missing = emit()
+    expect(missing.last_parts.traffic).toEqual(before.last_parts.traffic)
+    expect(JSON.parse(missing.pending[1])).not.toHaveProperty('packets_dropped')
+    v6(1); const recovered = emit()
+    const reports = recovered.pending.map((bytes: string) => JSON.parse(bytes))
+    expect(reports[2].packets_dropped).toBe(21)
+    expect(reports[2].packets_accepted).toBe(0)
+    expect(reports[2].counter_epoch.traffic).not.toBe(reports[0].counter_epoch.traffic)
+    expect(recovered.last_parts.traffic['v6/-A EVOFW_INPUT -m set --match-set evofw_deny_v6 src -j DROP']).toBe(1)
+  })
+
+})

apps/api/src/services/agent-hardening-mikrotik.test.ts

@@ -0,0 +1,141 @@
+import { linkEnrollmentCredential } from './enrollment.js'
+import { readFileSync } from 'node:fs'
+import { afterAll, describe, expect, it } from 'vitest'
+import { applyReportV2BodySchema } from '@evofw/shared'
+import { repos } from '@evofw/db'
+import { hashToken } from '../plugins/auth.js'
+import { buildApp } from '../app.js'
+import type { AppConfig } from '../config.js'
+const installer = readFileSync(new URL('../agent-scripts/mikrotik-install.rsc', import.meta.url), 'utf8')
+const config: AppConfig = { databaseUrl: 'sqlite::memory:', jwtSecret: 'test', jwtTtlHours: 24, serverPort: 8080, staticDir: null, logLevel: 'error', authRequired: false, authIssuer: 'https://auth.test', authPortalUrl: 'http://localhost:5175', publicBaseUrl: 'https://fw.example.com', enrollSeed: 'test-seed', corsOrigins: [], authAuditIngestSecret: null, secretKey: null, statsRetentionDays: 30 }
+
+describe('RouterOS dual-stack endpoint and imported-metadata contracts (no RouterOS runtime)', () => {
+  const appPromise = buildApp({ memory: true, config })
+  afterAll(async () => (await appPromise).close())
+  it('ACKs actual compiled hash/revision after a JSON-to-RSC generation race', async () => {
+    const app = await appPromise
+    const link = await app.inject({ method: 'POST', url: '/api/v1/install-links', payload: { name: 'rsc-race', platform: 'mikrotik' } })
+    expect(link.statusCode).toBe(201)
+    const { id, agent_id: agentId } = link.json()
+    const token = 'evofw_rsc_dualstack_race_token_12345678'
+    expect((await app.inject({ method: 'POST', url: '/v1/agent/enroll', headers: { 'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, id)!, config.enrollSeed) }, payload: { name: 'rsc-race', platform: 'mikrotik', token, install_link_id: id } })).statusCode).toBe(201)
+    await app.inject({ method: 'POST', url: `/api/v1/agents/${agentId}/approve` })
+    const headers = { authorization: `Bearer ${token}` }
+    const before = await app.inject({ method: 'GET', url: '/v1/agent/policy', headers })
+    expect(before.statusCode).toBe(200)
+    expect((await app.inject({ method: 'POST', url: `/api/v1/agents/${agentId}/overrides`, payload: { action: 'deny', cidr: '2001:db8:7::99/64' } })).statusCode).toBe(200)
+    const compiled = await app.inject({ method: 'GET', url: '/v1/agent/policy.rsc', headers })
+    expect(compiled.statusCode).toBe(200)
+    const renderedHash = compiled.body.match(/^:set EvofwRenderedHash "(sha256:[a-f0-9]{64})"$/m)?.[1]
+    const renderedRevision = Number(compiled.body.match(/^:set EvofwRenderedRevision ([0-9]+)$/m)?.[1])
+    expect(renderedHash).toBeTruthy()
+    expect(renderedHash).not.toBe(before.json().hash)
+    expect(renderedRevision).toBeGreaterThan(before.json().generation)
+    expect(compiled.body).toContain('/ipv6 firewall address-list add list=$deny6 address=2001:db8:7::/64')
+    const importAt = installer.indexOf('/import file-name=$file')
+    const importAcknowledgement = installer.slice(importAt, installer.indexOf(':set syncOk 1', importAt))
+    expect(installer.lastIndexOf(':set EvofwRenderedHash ""', importAt)).toBeGreaterThan(0)
+    expect(importAcknowledgement).toContain(':set EvofwLastHash $EvofwRenderedHash')
+    expect(importAcknowledgement).toContain(':set EvofwAppliedRevision $EvofwRenderedRevision')
+    expect(importAcknowledgement).not.toContain(':set EvofwLastHash $hash')
+    const report = { telemetry_version: 2, report_id: 'rsc-race-ack', sequence: 1, boot_id: 'rsc-boot', observed_at: new Date().toISOString(), interval_ms: 60000, counter_epoch: { traffic: 'rsc-epoch' }, agent_version: 'rsc/9', source: 'mikrotik', status: 'ok', kernel_method: 'address-list', applied_hash: renderedHash, applied_revision: renderedRevision, capabilities: ['telemetry-v2', 'policy-ipv4', 'policy-ipv6', 'mikrotik-staged-apply'], quality: { partial: true, truncated: false, unsupported: ['ip_counters_presence_only', 'port_counters_unavailable'], lost_reports: 0 } }
+    expect(applyReportV2BodySchema.safeParse(report).success).toBe(true)
+    const ack = await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: report })
+    expect(ack.statusCode, ack.body).toBe(200)
+    expect(ack.json()).toMatchObject({ ok: true, accepted: true, report_id: report.report_id, sequence: 1 })
+    const agent = await app.inject({ method: 'GET', url: `/api/v1/agents/${agentId}` })
+    expect(agent.json().diagnostics).toMatchObject({ applied_hash: renderedHash, applied_revision: renderedRevision, status: 'in_sync' })
+  })
+  it('retains final-snapshot pending payload on failed ACK and gates mutation on capability before import', () => {
+    const policyFetch = installer.indexOf('/v1/agent/policy?transport=routeros")')
+    const unsupported = installer.indexOf('IPv6 enforcement unavailable', policyFetch)
+    const importAt = installer.indexOf('/import file-name=$file')
+    expect(unsupported).toBeGreaterThan(policyFetch)
+    expect(unsupported).toBeLessThan(importAt)
+    expect(installer).toContain(':set EvofwApplyErrorCode "UNSUPPORTED_CAPABILITY"')
+    expect(installer).toContain('final telemetry ACK pending; defer counter-resetting apply')
+    expect(installer).toContain(':if ([:len ($telemetryState->"pending")] > 0) do={ :return }')
+    expect(installer.indexOf('contents=[:serialize $telemetryState', installer.indexOf(':local final '))).toBeLessThan(importAt)
+    expect(installer).not.toContain('/v1/agent/heartbeat')
+  })
+  it('aggregates both families conservatively and never converts address presence into packets', () => {
+    expect(installer).toContain(':local readCounters do={')
+    expect(installer).toContain('/ipv6 firewall filter get $id packets')
+    expect(installer).toContain('} on-error={ :set valid false }')
+    expect(installer).toContain(':if (!$valid) do={ :return {"valid"=false} }')
+    expect(installer).toContain(':if (($counters->"valid") = true)')
+    expect(installer).toContain('traffic_counters_unavailable')
+    expect(installer).toContain('"policy-ipv6";"traffic-counters-ipv6"')
+    expect(installer).not.toContain('"ip_hits"=')
+    expect(installer).not.toContain('"port_hits"=')
+  })
+  it('rejects a narrowed default DROP in each family before declaring a verified no-op', () => {
+    const drift = installer.slice(installer.indexOf(':local drift 0'), installer.indexOf(':if ($hash = $EvofwLastHash'))
+    for (const family of ['ip', 'ipv6']) {
+      const start = drift.indexOf(`:local rule [/${family} firewall filter find comment=evofw-default-drop-forward]`)
+      const block = drift.slice(start, drift.indexOf(':local ordered', start))
+      expect(block).toContain(`:if ([/${family} firewall filter get $rule src-address-list] != "") do={ :error "owned default matcher drift" }`)
+      const permitted = block.match(/:local permitted \{([^}]+)\}/)?.[1]
+      for (const narrowing of ['src-address', 'dst-address', 'dst-address-list', 'protocol', 'dst-port', 'in-interface', 'connection-state']) {
+        expect(permitted).not.toContain(`"${narrowing}"`)
+      }
+      expect(block).toContain('owned matcher drift')
+    }
+  })
+  it('retains complete per-rule reset baselines across unavailable observations and shares detection with final snapshots', () => {
+    const read = installer.slice(installer.indexOf(':local readCounters do={'), installer.indexOf(':local syncOk 0'))
+    for (const family of ['ip', 'ipv6']) {
+      for (const rule of ['deny-drop-input', 'deny-drop-forward', 'default-drop-forward', 'allow-accept-forward']) {
+        expect(read).toContain(`:set ($observed->"${family}:evofw-${rule}") $n`)
+      }
+    }
+    // A single reset is visible even if another family or rule grows faster.
+    expect(read).toContain(':foreach key,value in=$observed do={')
+    expect(read).toContain(':local previous ($EvofwRuleCounters->$key)')
+    expect(read).toContain('$value < $previous')
+    expect(read.indexOf(':if (!$valid)')).toBeLessThan(read.indexOf(':set EvofwRuleCounters $observed'))
+    expect(read).toContain(':if ($reset) do={ :set EvofwTrafficEpoch [:rndstr length=32] }')
+    expect(installer).not.toContain('EvofwLastDropped')
+    expect(installer).not.toContain('EvofwLastAccepted')
+    const finalStart = installer.indexOf(':local finalCounters [$readCounters]')
+    expect(finalStart).toBeLessThan(installer.indexOf(':local final {'))
+    expect(installer).toContain('($finalCounters->"reset") = true')
+    expect(installer).toContain('($counters->"reset") = true')
+    expect(installer.match(/counter_reset_coverage_unknown/g)).toHaveLength(2)
+    const importAt = installer.indexOf('/import file-name=$file')
+    expect(installer.indexOf(':set EvofwRuleCounters [:toarray ""]', importAt)).toBeGreaterThan(importAt)
+  })
+  it('distinguishes the opt-in budget envelope from valid policy and authentication failure before import', async () => {
+    const app = await appPromise
+    const token = 'rsc-budget-transport-token'
+    repos.insertAgent(app.db, { id: 'rsc-budget', name: 'rsc-budget', tokenPrefix: 'rsc', tokenHash: hashToken(token), status: 'approved' })
+    repos.ensureSharedSetAssigned(app.db, 'rsc-budget')
+    const headers = { authorization: `Bearer ${token}` }
+    const valid = await app.inject({ url: '/v1/agent/policy?transport=routeros', headers })
+    expect(valid.statusCode).toBe(200); expect(valid.json()).toHaveProperty('hash'); expect(valid.json()).not.toHaveProperty('error')
+    const insert = app.sqlite.prepare("INSERT INTO agent_port_rules(id,agent_id,action,protocol,port_start,port_end,src_kind) VALUES(?,'rsc-budget','open','both',80,80,'all')")
+    app.sqlite.transaction(() => { for (let i = 0; i < 501; i++) insert.run(`rsc-budget-${i}`) })()
+    repos.bumpAgentGeneration(app.db, 'rsc-budget')
+    const rejected = await app.inject({ url: '/v1/agent/policy?transport=routeros', headers })
+    expect(rejected.statusCode).toBe(200); expect(rejected.json().error.code).toBe('POLICY_BUDGET_EXCEEDED')
+    expect(rejected.json()).not.toHaveProperty('hash'); expect(rejected.json()).not.toHaveProperty('generation')
+    expect(rejected.headers.etag).toBeUndefined(); expect(rejected.headers['cache-control']).toBe('no-store')
+    expect((await app.inject({ url: '/v1/agent/policy?transport=routeros', headers: { authorization: 'Bearer wrong' } })).statusCode).toBe(401)
+    const envelopeAt = installer.indexOf(':if ([:typeof ($p->"error")] = "array")')
+    expect(envelopeAt).toBeGreaterThan(0)
+    expect(envelopeAt).toBeLessThan(installer.indexOf(':local hash ($p->"hash")'))
+    expect(installer).toContain('unexpected policy error envelope')
+    expect(installer).not.toContain('[:typeof ($p->"error")] != "nothing"')
+    expect(installer.indexOf('policy fetch failed')).toBeLessThan(envelopeAt)
+  })
+  it('keeps native stale-floor recovery bounded and selects persistent flash storage when present', () => {
+    expect(installer).toContain('$flashType = "disk" || $flashType = "directory"')
+    expect(installer).toContain('flash/evofw-telemetry.json')
+    expect(installer).toContain('$floor < ($expected->"sequence") || $floor >= 9007199254740991')
+    expect(installer).toContain(':set ($state->"next_sequence") ($EvofwSequenceFloor + 1)')
+    expect(installer).toContain(':set ($state->"coverage_unknown") true')
+    expect(installer).toContain(':set EvofwTrafficEpoch [:rndstr length=32]')
+    expect(installer).toContain('sequence_rollback_coverage_unknown')
+    expect(installer.match(/\$resyncSequence state=\$telemetryState/g)).toHaveLength(3)
+  })
+})

apps/api/src/services/agent-hardening-telemetry.test.ts

@@ -0,0 +1,317 @@
+import { repos } from '@evofw/db'
+import { linkEnrollmentCredential } from './enrollment.js'
+import { spawnSync } from 'node:child_process'
+import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { afterAll, afterEach, describe, expect, it } from 'vitest'
+import { applyReportV2BodySchema } from '@evofw/shared'
+import { buildApp } from '../app.js'
+import type { AppConfig } from '../config.js'
+
+const source = readFileSync(new URL('../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8')
+const python = process.platform === 'win32' ? 'python' : 'python3'
+const bash = process.platform === 'win32' ? 'C:/Program Files/Git/bin/bash.exe' : 'bash'
+const dirs: string[] = []
+function workspace() { const dir = mkdtempSync(join(tmpdir(), 'evofw-v2-test-')); dirs.push(dir); return dir }
+afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }) })
+function embedded(name: string) { const s = source.slice(source.indexOf(`${name}() {`)); const begin = s.indexOf("<<'PY'") + 7; return s.slice(begin, s.indexOf('\nPY', begin)).trimStart() }
+function worker(dir: string, op: string, ...args: string[]) {
+  return spawnSync(python, ['-c', embedded('telemetry_worker'), dir, dir, op, ...args], { encoding: 'utf8', env: { ...process.env, EVOFW_TEST_BOOT_ID: 'fixture-boot' } })
+}
+function state(dir: string) { return JSON.parse(readFileSync(join(dir, 'telemetry-state.json'), 'utf8')) }
+function pending(dir: string) { return JSON.parse(state(dir).pending[0]) }
+function initialize() { const dir = workspace(); const r = worker(dir, 'init'); expect(r.status, r.stderr).toBe(0); return dir }
+function rule(chain: string, id: string, verdict: 'drop' | 'accept', packets: number) { return { rule: { family: 'inet', table: 'evofw', chain, comment: `evofw-v2/${id}`, expr: [{ counter: { packets, bytes: 0 } }, { [verdict]: null }] } } }
+function snapshot(dir: string, entries: unknown[]) { writeFileSync(join(dir, 'metrics-nft.json'), JSON.stringify({ nftables: entries })); const r = worker(dir, 'capture', 'nft'); expect(r.status, r.stderr).toBe(0) }
+function queue(dir: string) { const r = worker(dir, 'enqueue', 'ok', 'nft'); expect(r.status, r.stderr).toBe(0); return pending(dir) }
+
+const config: AppConfig = { databaseUrl: 'sqlite::memory:', jwtSecret: 'test', jwtTtlHours: 24, serverPort: 8080, staticDir: null, logLevel: 'error', authRequired: false, authIssuer: 'https://auth.test', authPortalUrl: 'http://localhost:5175', publicBaseUrl: 'https://fw.example.com', enrollSeed: 'test-seed', corsOrigins: [], authAuditIngestSecret: null, secretKey: null, statsRetentionDays: 30 }
+
+describe('agent hardening real endpoint contracts', () => {
+  const appPromise = buildApp({ memory: true, config })
+  afterAll(async () => (await appPromise).close())
+  let enrollment = 0
+  async function enroll(platform: 'linux' | 'mikrotik') {
+    const app = await appPromise
+    const token = `agent-hardening-${platform}-token-${++enrollment}-12345678`
+    const link = await app.inject({ method: 'POST', url: '/api/v1/install-links', payload: { name: `hardening-${platform}-${enrollment}`, platform } })
+    expect(link.statusCode).toBe(201)
+    const { id, agent_id: agentId } = link.json()
+    const enrolled = await app.inject({ method: 'POST', url: '/v1/agent/enroll', headers: { 'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, id)!, config.enrollSeed) }, payload: { name: `hardening-${platform}-${enrollment}`, platform, token, install_link_id: id } })
+    expect(enrolled.statusCode).toBe(201)
+    await app.inject({ method: 'POST', url: `/api/v1/agents/${agentId}/approve` })
+    return { app, token, agentId, headers: { authorization: `Bearer ${token}` } }
+  }
+  it('passes actual API policy through the Bash parser and checked nft apply arguments', async () => {
+    const { app, headers, agentId } = await enroll('linux')
+    const set = await app.inject({ method: 'POST', url: '/api/v1/policy-sets', payload: { name: 'agent-contract-set' } })
+    expect(set.statusCode).toBe(200)
+    for (const [action, cidr] of [['deny', '10.23.1.9/24'], ['allow', '192.0.2.9/32']]) {
+      const rule = await app.inject({ method: 'POST', url: '/api/v1/rules', payload: { set_id: set.json().id, action, cidr } })
+      expect(rule.statusCode).toBe(200)
+    }
+    expect((await app.inject({ method: 'PUT', url: `/api/v1/agents/${agentId}/policy-sets`, payload: { set_ids: [set.json().id] } })).statusCode).toBe(200)
+    expect((await app.inject({ method: 'POST', url: `/api/v1/agents/${agentId}/port-rules`, payload: { action: 'open', protocol: 'both', port_start: 443, src_kind: 'cidr', src_cidr: '192.0.2.0/24' } })).statusCode).toBe(200)
+    const response = await app.inject({ method: 'GET', url: '/v1/agent/policy', headers })
+    expect(response.statusCode).toBe(200)
+    const actual = response.json()
+    expect(actual.hash).toMatch(/^sha256:[0-9a-f]{64}$/)
+    expect(actual.deny_cidrs).toEqual(['10.23.1.0/24'])
+    expect(actual.port_rules).toHaveLength(2)
+    const dir = workspace(); writeFileSync(join(dir, 'policy.json'), response.body)
+    const pythonPath = spawnSync(python, ['-c', 'import sys;print(sys.executable)'], { encoding: 'utf8' }).stdout.trim().replaceAll('\\', '/')
+    const parse = source.slice(source.indexOf('parse_policy() {'), source.indexOf('HASH="";'))
+    const render = source.slice(source.indexOf('render_nft_batch() {'), source.indexOf('# Collect observed host firewall'))
+    writeFileSync(join(dir, 'test.sh'), `set -euo pipefail\nRUN_DIR=.; LOG_FILE=log; PORT_RULES_FILE=./port_rules.json; DENY=(); ALLOW=();\npython3() { '${pythonPath}' "$@"; }\nlog() { :; }\nnft() { printf '%s\\n' "$*" >>calls; if [[ "$1" == '--check' ]]; then cp "$3" checked.batch; elif [[ "$1" == '--file' ]]; then cmp checked.batch "$2"; fi; }\n${parse}\n${render}\nparse_policy policy.json\napply_nft\nprintf '%s\\n' "$HASH" "$DEFAULT_ACTION" "$APPLIED" > parsed.txt\n`)
+    const run = spawnSync(bash, ['--noprofile', '--norc', join(dir, 'test.sh').replaceAll('\\', '/')], { cwd: dir, encoding: 'utf8' })
+    expect(run.status, run.stderr).toBe(0)
+    const lines = readFileSync(join(dir, 'parsed.txt'), 'utf8').trim().split(/\r?\n/)
+    expect(lines).toEqual([actual.hash, actual.default_action, String(actual.deny_cidrs.length + actual.allow_cidrs.length)])
+    expect(readFileSync(join(dir, 'calls'), 'utf8')).toContain('--check --file ./nft.batch')
+    expect(readFileSync(join(dir, 'checked.batch'), 'utf8')).toContain('add element inet evofw deny_v4 { 10.23.1.0/24 }')
+    expect(readFileSync(join(dir, 'checked.batch'), 'utf8')).toContain('forward ct status dnat ip saddr @port_src_0_v4 tcp dport 443')
+  })
+  it('acknowledges a newer generation with identical hash after verified no-op without changing epochs or pending bytes', async () => {
+    const { app, headers, agentId } = await enroll('linux')
+    const firstResponse = await app.inject({ method: 'GET', url: '/v1/agent/policy', headers })
+    const first = firstResponse.json()
+    const dir = initialize()
+    writeFileSync(join(dir, 'last_policy.json'), firstResponse.body)
+    snapshot(dir, [rule('input', 'input/drop', 'drop', 7)])
+    queue(dir)
+    const original = state(dir)
+    const bumped = await app.inject({ method: 'PUT', url: `/api/v1/agents/${agentId}/policy-sets`, payload: { set_ids: [] } })
+    expect(bumped.statusCode).toBe(200)
+    const nextResponse = await app.inject({ method: 'GET', url: '/v1/agent/policy', headers })
+    const next = nextResponse.json()
+    expect(next.hash).toBe(first.hash)
+    expect(next.generation).toBeGreaterThan(first.generation)
+    writeFileSync(join(dir, 'candidate.json'), nextResponse.body)
+    writeFileSync(join(dir, 'last_hash'), first.hash)
+    writeFileSync(join(dir, 'kernel_state'), 'nft verified\n')
+    const pythonPath = spawnSync(python, ['-c', 'import sys;print(sys.executable)'], { encoding: 'utf8' }).stdout.trim().replaceAll('\\', '/')
+    const workerFunctions = source.slice(source.indexOf('telemetry_worker() {'), source.indexOf('telemetry_worker init'))
+    const parse = source.slice(source.indexOf('parse_policy() {'), source.indexOf('HASH="";'))
+    const noChange = source.slice(source.indexOf('KERNEL_STATE_FILE="$STATE_DIR/kernel_state"'), source.indexOf('# Durably queue the final old-epoch'))
+    writeFileSync(join(dir, 'no-change.sh'), `set -euo pipefail\nSTATE_DIR=.; RUN_DIR=.; HASH_FILE=last_hash; BACKEND=nft; DENY=(); ALLOW=();\npython3() { '${pythonPath}' "$@"; }\nlog() { :; }\nkernel_digest() { echo verified; }\napply_nft() { echo unexpected >> applied; }\n${workerFunctions}\n${parse}\nsend_report() { telemetry_worker enqueue ok nft; }\nparse_policy candidate.json\n${noChange}`)
+    const result = spawnSync(bash, ['--noprofile', '--norc', join(dir, 'no-change.sh').replaceAll('\\', '/')], { cwd: dir, encoding: 'utf8' })
+    expect(result.status, result.stderr).toBe(0)
+    const updated = state(dir)
+    expect(updated.epochs).toEqual(original.epochs)
+    expect(updated.pending[0]).toBe(original.pending[0])
+    expect(JSON.parse(updated.pending[1]).applied_revision).toBe(next.generation)
+    expect(JSON.parse(readFileSync(join(dir, 'last_policy.json'), 'utf8')).generation).toBe(next.generation)
+    expect(() => readFileSync(join(dir, 'applied'))).toThrow()
+    for (const bytes of updated.pending) {
+      const response = await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: JSON.parse(bytes) })
+      expect(response.statusCode, response.body).toBe(200)
+    }
+    const agent = await app.inject({ method: 'GET', url: `/api/v1/agents/${agentId}` })
+    expect(agent.json().diagnostics.applied_revision).toBe(next.generation)
+    expect(agent.json().diagnostics.status).toBe('in_sync')
+  })
+  it('compiles mixed IPv4/IPv6 policy from the real API without cross-family ACL bypass', async () => {
+    const { app, headers, agentId } = await enroll('linux')
+    for (const cidr of ['192.0.2.1/32', '2001:db8:1::99/64']) {
+      const response = await app.inject({ method: 'POST', url: `/api/v1/agents/${agentId}/overrides`, payload: { action: 'deny', cidr } })
+      expect(response.statusCode).toBe(200)
+    }
+    const port = await app.inject({ method: 'POST', url: `/api/v1/agents/${agentId}/port-rules`, payload: { action: 'open', protocol: 'tcp', port_start: 443, src_kind: 'cidr', src_cidr: '2001:db8:2::/64' } })
+    expect(port.statusCode).toBe(200)
+    const response = await app.inject({ method: 'GET', url: '/v1/agent/policy', headers })
+    expect(response.statusCode).toBe(200)
+    const dir = workspace(); writeFileSync(join(dir, 'candidate.json'), response.body)
+    const parsed = spawnSync(python, ['-c', embedded('parse_policy'), join(dir, 'candidate.json'), dir], { encoding: 'utf8' })
+    expect(parsed.status, parsed.stderr).toBe(0)
+    const rendered = spawnSync(python, ['-c', embedded('render_nft_batch'), join(dir, 'normalized.json'), '1', '1', '1'], { encoding: 'utf8' })
+    expect(rendered.status, rendered.stderr).toBe(0)
+    expect(rendered.stdout).toContain('add element inet evofw deny_v4 { 192.0.2.1/32 }')
+    expect(rendered.stdout).toContain('add element inet evofw deny_v6 { 2001:db8:1::/64 }')
+    expect(rendered.stdout).toContain('forward ct status dnat ip6 saddr @port_src_0_v6 tcp dport 443 counter accept')
+    expect(rendered.stdout).not.toContain('ip saddr @port_src_0_v6')
+  })
+  it('renders staged RouterOS script from actual authenticated policy.rsc endpoint', async () => {
+    const { app, headers } = await enroll('mikrotik')
+    const response = await app.inject({ method: 'GET', url: '/v1/agent/policy.rsc', headers })
+    expect(response.statusCode).toBe(200)
+    expect(response.body).toContain(':local deny4 ("EVOFW_DENY_" . $tx)')
+    expect(response.body).toContain('owned filter chain/action drift')
+    expect(response.body).toContain('owned filter order drift')
+    expect(response.body).toContain('/ip firewall filter remove $new4')
+  })
+  it('roundtrips Python-emitted v2 report through the shared schema, ingest, ACK and replay', async () => {
+    const { app, headers } = await enroll('linux')
+    const dir = initialize()
+    const policy = await app.inject({ method: 'GET', url: '/v1/agent/policy', headers })
+    writeFileSync(join(dir, 'last_policy.json'), policy.body)
+    snapshot(dir, [rule('input', 'input/drop', 'drop', 10)])
+    const report = queue(dir)
+    expect(applyReportV2BodySchema.safeParse(report).success).toBe(true)
+    const accepted = await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: report })
+    expect(accepted.statusCode, accepted.body).toBe(200)
+    expect(accepted.json()).toMatchObject({ ok: true, accepted: true, report_id: report.report_id, sequence: report.sequence })
+    const replay = await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: report })
+    expect(replay.json()).toMatchObject({ ok: true, duplicate: true, report_id: report.report_id })
+    writeFileSync(join(dir, 'report-ack.json'), replay.body)
+    expect(worker(dir, 'ack').status).toBe(0)
+    expect(state(dir).pending).toHaveLength(0)
+  })
+  it('recovers rolled-back durable sequence through an authenticated API floor without replaying invented deltas', async () => {
+    const { app, headers } = await enroll('linux')
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5)])
+    const old = queue(dir); const immutable = state(dir).pending[0]
+    const newer = { ...old, report_id: 'server-already-accepted-100', sequence: 100 }
+    expect((await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: newer })).json()).toMatchObject({ accepted: true, sequence_floor: 100 })
+    const stale = await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: old })
+    expect(stale.statusCode, stale.body).toBe(200)
+    expect(stale.json()).toMatchObject({ reason: 'stale_sequence', sequence_floor: 100, report_id: old.report_id, sequence: 0 })
+    expect(state(dir).pending[0]).toBe(immutable)
+    writeFileSync(join(dir, 'report-ack.json'), stale.body)
+    expect(worker(dir, 'ack').status).toBe(0)
+    expect(state(dir)).toMatchObject({ next_sequence: 101, coverage_unknown: true, last_parts: {} })
+    snapshot(dir, [rule('input', 'input/drop', 'drop', 8)])
+    const recovered = queue(dir)
+    expect(recovered.sequence).toBe(101)
+    expect(recovered.report_id).not.toBe(old.report_id)
+    expect(recovered.counter_epoch.traffic).not.toBe(old.counter_epoch.traffic)
+    expect(recovered.quality).toMatchObject({ partial: true })
+    expect(recovered.quality.unsupported).toContain('sequence_rollback_coverage_unknown')
+    expect(applyReportV2BodySchema.safeParse(recovered).success).toBe(true)
+    expect((await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload: recovered })).json()).toMatchObject({ accepted: true, sequence_floor: 101 })
+  })
+})
+
+describe('agent hardening telemetry v2 durable state and collectors', () => {
+  it('does not double-count prerouting accepts or fabricate missing metrics', () => {
+    const dir = initialize()
+    snapshot(dir, [rule('prerouting', 'pre/accept', 'accept', 100), rule('prerouting', 'pre/drop', 'drop', 4), rule('input', 'input/accept', 'accept', 8), rule('forward', 'forward/drop', 'drop', 3)])
+    const report = queue(dir)
+    expect(report.packets_accepted).toBe(8)
+    expect(report.packets_dropped).toBe(7)
+    expect(report).not.toHaveProperty('ip_hits')
+    expect(report.quality.unsupported).toContain('ip_counters_unavailable')
+    expect(applyReportV2BodySchema.safeParse(report).success).toBe(true)
+  })
+  it('keeps queued bytes and ID stable across restart and rejects mismatched ACK', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5)]); const report = queue(dir)
+    const bytes = state(dir).pending[0]
+    expect(worker(dir, 'init').status).toBe(0)
+    expect(state(dir).pending[0]).toBe(bytes)
+    writeFileSync(join(dir, 'report-ack.json'), JSON.stringify({ ok: true, accepted: true, report_id: 'wrong', sequence: report.sequence }))
+    expect(worker(dir, 'ack').status).not.toBe(0)
+    expect(state(dir).pending[0]).toBe(bytes)
+    writeFileSync(join(dir, 'report-ack.json'), JSON.stringify({ ok: true, accepted: true, report_id: report.report_id, sequence: report.sequence }))
+    expect(worker(dir, 'ack').status).toBe(0)
+    queue(dir)
+    expect(pending(dir).sequence).toBe(report.sequence + 1)
+  })
+  it('refuses malformed stale floors and stops sync at the authenticated maximum without discarding pending bytes', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5)]); const report = queue(dir)
+    const original = readFileSync(join(dir, 'telemetry-state.json'), 'utf8')
+    for (const floor of [undefined, -1, 1.5, '100', Number.MAX_SAFE_INTEGER + 1]) {
+      writeFileSync(join(dir, 'report-ack.json'), JSON.stringify({ ok: true, reason: 'stale_sequence', report_id: report.report_id, sequence: report.sequence, sequence_floor: floor }))
+      expect(worker(dir, 'ack').status).not.toBe(0)
+      expect(readFileSync(join(dir, 'telemetry-state.json'), 'utf8')).toBe(original)
+    }
+    writeFileSync(join(dir, 'report-ack.json'), JSON.stringify({ ok: true, reason: 'stale_sequence', report_id: report.report_id, sequence: report.sequence, sequence_floor: Number.MAX_SAFE_INTEGER }))
+    expect(worker(dir, 'ack').status).not.toBe(0)
+    expect(state(dir).pending).toEqual(JSON.parse(original).pending)
+    expect(state(dir).sequence_exhausted).toBe(true)
+    expect(worker(dir, 'assert-sequence').status).not.toBe(0)
+    expect(worker(dir, 'enqueue', 'ok', 'nft').status).not.toBe(0)
+    const startup = source.slice(source.indexOf('telemetry_worker init'), source.indexOf('curl_policy() {'))
+    expect(startup.indexOf('flush_reports')).toBeLessThan(startup.indexOf('telemetry_worker assert-sequence'))
+    expect(startup.indexOf('telemetry_worker assert-sequence')).toBeLessThan(startup.indexOf('maybe_self_update'))
+  })
+  it('detects counter reset with an epoch transition, preserving queued old epoch', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 50)]); queue(dir)
+    snapshot(dir, [rule('input', 'input/drop', 'drop', 2)]); queue(dir)
+    const reports = state(dir).pending.map((s: string) => JSON.parse(s))
+    expect(reports[1].counter_epoch.traffic).not.toBe(reports[0].counter_epoch.traffic)
+    expect(reports[1].sequence).toBe(reports[0].sequence + 1)
+    expect(reports[0].packets_dropped).toBe(50)
+  })
+  it('omits unsafe integer totals and labels degraded quality', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', Number.MAX_SAFE_INTEGER), rule('forward', 'forward/drop', 'drop', 1)])
+    const report = queue(dir)
+    expect(report).not.toHaveProperty('packets_dropped')
+    expect(report).not.toHaveProperty('packets_accepted')
+    expect(report.quality.unsupported).toContain('traffic_counter_unsafe')
+    expect(applyReportV2BodySchema.safeParse(report).success).toBe(true)
+  })
+  it('parses measured host/port elements, omits prefix aggregates masquerading as hosts', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5),
+      { set: { family: 'inet', table: 'evofw', name: 'deny_v4', elem: [{ elem: { val: '192.0.2.1', counter: { packets: 5 } } }, { elem: { val: { prefix: { addr: '10.0.0.0', len: 24 } }, counter: { packets: 100 } } }] } },
+      { set: { family: 'inet', table: 'evofw', name: 'deny_port_hits', elem: [{ elem: { val: { concat: ['192.0.2.1', 'tcp', 443] }, counter: { packets: 4 } } }] } },
+    ])
+    const report = queue(dir)
+    expect(report.ip_hits).toEqual([{ ip: '192.0.2.1', packets: 5 }])
+    expect(report.port_hits).toEqual([{ ip: '192.0.2.1', port: 443, protocol: 'tcp', packets: 4 }])
+    expect(report.detail_counts).toEqual({ ip_found: 1, ip_sent: 1, port_found: 1, port_sent: 1 })
+    expect(report.quality.unsupported).toContain('ip_prefix_counters')
+    expect(applyReportV2BodySchema.safeParse(report).success).toBe(true)
+  })
+  it('bounds the queue and preserves every pending report when full', () => {
+    const dir = initialize(); const before = state(dir)
+    before.pending = Array.from({ length: 32 }, () => '{"persisted":true}'); writeFileSync(join(dir, 'telemetry-state.json'), JSON.stringify(before))
+    expect(worker(dir, 'enqueue', 'ok', 'nft').status).not.toBe(0)
+    expect(state(dir).pending).toEqual(before.pending)
+    expect(state(dir).lost_reports).toBe(1)
+  })
+  it('rotates boot epochs without rewriting pending old-boot reports', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5)]); const report = queue(dir)
+    const result = spawnSync(python, ['-c', embedded('telemetry_worker'), dir, dir, 'init'], { encoding: 'utf8', env: { ...process.env, EVOFW_TEST_BOOT_ID: 'next-boot' } })
+    expect(result.status, result.stderr).toBe(0)
+    expect(pending(dir)).toEqual(report)
+    expect(state(dir).epochs.traffic).not.toBe(report.counter_epoch.traffic)
+    expect(state(dir).next_sequence).toBe(report.sequence + 1)
+  })
+  it('does not allocate a sequence or lose pending reports if persistence fails', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5)]); queue(dir)
+    const before = readFileSync(join(dir, 'telemetry-state.json'), 'utf8')
+    mkdirSync(join(dir, 'telemetry-state.json.new'))
+    expect(worker(dir, 'enqueue', 'ok', 'nft').status).not.toBe(0)
+    expect(readFileSync(join(dir, 'telemetry-state.json'), 'utf8')).toBe(before)
+  })
+  it('keeps the exact pending report after a simulated HTTP timeout', () => {
+    const dir = initialize(); snapshot(dir, [rule('input', 'input/drop', 'drop', 5)]); queue(dir)
+    const before = state(dir).pending[0]
+    const workerFunction = source.slice(source.indexOf('telemetry_worker() {'), source.indexOf('telemetry_worker init'))
+    const pythonPath = spawnSync(python, ['-c', 'import sys;print(sys.executable)'], { encoding: 'utf8' }).stdout.trim().replaceAll('\\', '/')
+    writeFileSync(join(dir, 'timeout.sh'), `set -euo pipefail\nSTATE_DIR=.; RUN_DIR=.; EVOFW_CP_URL=https://unused.invalid; CLIENT_TOKEN=fixture;\npython3() { '${pythonPath}' "$@"; }\ncurl() { return 28; }\nlog() { :; }\n${workerFunction}\nflush_reports\n`)
+    const result = spawnSync(bash, ['--noprofile', '--norc', join(dir, 'timeout.sh').replaceAll('\\', '/')], { cwd: dir, encoding: 'utf8' })
+    expect(result.status, result.stderr).toBe(0)
+    expect(state(dir).pending[0]).toBe(before)
+  })
+  it.each([true, false])('queues the final epoch before apply; queue success=%s', (success) => {
+    const dir = workspace(); writeFileSync(join(dir, 'last_hash'), 'previous'); writeFileSync(join(dir, 'kernel_state'), 'nft digest')
+    const block = source.slice(source.indexOf('# Durably queue the final old-epoch'), source.indexOf('# Store a verified structural digest'))
+    writeFileSync(join(dir, 'order.sh'), `set -euo pipefail\nHASH_FILE=last_hash; KERNEL_STATE_FILE=kernel_state; BACKEND=nft;\ncollect_telemetry() { echo snapshot >> calls; }\ntelemetry_worker() { echo "$1" >> calls; ${success ? ':' : 'return 1'}; }\nflush_reports() { echo flush >> calls; }\ntype() { return 0; }\napply_nft() { echo apply >> calls; }\n${block}`)
+    const result = spawnSync(bash, ['--noprofile', '--norc', join(dir, 'order.sh').replaceAll('\\', '/')], { cwd: dir, encoding: 'utf8' })
+    const calls = readFileSync(join(dir, 'calls'), 'utf8').trim().split(/\r?\n/)
+    expect(calls).toEqual(success ? ['snapshot', 'enqueue', 'flush', 'apply'] : ['snapshot', 'enqueue'])
+    expect(result.status).toBe(success ? 0 : 1)
+  })
+
+  it('drains loss counts once per durable enqueue: one loss then reports 1,0,0', () => {
+    const dir = initialize()
+    const accumulated = state(dir); accumulated.lost_reports = 1
+    writeFileSync(join(dir, 'telemetry-state.json'), JSON.stringify(accumulated))
+    snapshot(dir, [rule('input', 'input/drop', 'drop', 5)])
+    queue(dir); queue(dir); queue(dir)
+    const counts = state(dir).pending.map((bytes: string) => JSON.parse(bytes).quality.lost_reports)
+    expect(counts).toEqual([1, 0, 0])
+    expect(counts.reduce((a: number, b: number) => a + b, 0)).toBe(1)
+    expect(state(dir).lost_reports).toBe(0)
+  })
+  it('retains accumulated loss when the durable enqueue write fails', () => {
+    const dir = initialize(); const accumulated = state(dir); accumulated.lost_reports = 1
+    writeFileSync(join(dir, 'telemetry-state.json'), JSON.stringify(accumulated))
+    mkdirSync(join(dir, 'telemetry-state.json.new'))
+    expect(worker(dir, 'enqueue', 'ok', 'nft').status).not.toBe(0)
+    expect(state(dir).lost_reports).toBe(1)
+    expect(state(dir).pending).toEqual([])
+  })
+
+})

apps/api/src/services/agent-hardening.test.ts

@@ -0,0 +1,213 @@
+import { spawnSync } from 'node:child_process'
+import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { afterEach, describe, expect, it } from 'vitest'
+import { renderMikrotikPolicyRsc } from './policy/mikrotik-rsc.js'
+import type { EvaluatedPolicy } from './policy/evaluate.js'
+
+const script = readFileSync(new URL('../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8')
+const bash = process.platform === 'win32' ? 'C:/Program Files/Git/bin/bash.exe' : 'bash'
+const python = process.platform === 'win32' ? 'python' : 'python3'
+const dirs: string[] = []
+function workspace() { const dir = mkdtempSync(join(tmpdir(), 'evofw-agent-test-')); dirs.push(dir); return dir }
+afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }) })
+function embedded(name: string) {
+  const body = script.slice(script.indexOf(`${name}() {`))
+  const start = body.indexOf("<<'PY'") + 7
+  return body.slice(start, body.indexOf('\nPY', start)).trimStart()
+}
+const policy = (patch = {}) => ({ hash: 'sha256:' + 'a'.repeat(64), default_action: 'accept', deny_cidrs: ['10.0.0.1/24'], allow_cidrs: ['192.0.2.1'], port_rules: [], ...patch })
+function normalize(value: unknown) {
+  const dir = workspace(); const path = join(dir, 'policy.json'); writeFileSync(path, JSON.stringify(value))
+  return { dir, result: spawnSync(python, ['-c', embedded('parse_policy'), path, dir], { encoding: 'utf8' }) }
+}
+function render(value: unknown, counters = '1', hits = '1') {
+  const { dir, result } = normalize(value); expect(result.status, result.stderr).toBe(0)
+  return spawnSync(python, ['-c', embedded('render_nft_batch'), join(dir, 'normalized.json'), counters, hits, '1'], { encoding: 'utf8' })
+}
+const pythonExe = spawnSync(python, ['-c', 'import sys;print(sys.executable)'], { encoding: 'utf8' }).stdout.trim().replaceAll('\\', '/')
+function shell(content: string) {
+  content = `python3() { '${pythonExe}' \"$@\"; }\n` + content
+  const dir = workspace(); const path = join(dir, 'test.sh'); writeFileSync(path, content)
+  return { dir, result: spawnSync(bash, ['--noprofile', '--norc', path.replaceAll('\\', '/')], { cwd: dir, encoding: 'utf8' }) }
+}
+function fn(name: string, next: string) { return script.slice(script.indexOf(`${name === "apply_ipset" ? "prepare_ipset_rules" : name}() {`), script.indexOf(next, script.indexOf(`${name}() {`))) }
+
+describe('agent hardening policy boundary', () => {
+  it('normalizes and collapses IPv4 networks through data files', () => {
+    const { dir, result } = normalize(policy({ deny_cidrs: ['10.0.0.1/25', '10.0.0.128/25'] }))
+    expect(result.status, result.stderr).toBe(0)
+    expect(readFileSync(join(dir, 'deny'), 'utf8').replaceAll('\r', '')).toBe('10.0.0.0/24\n')
+    expect(readFileSync(join(dir, 'allow'), 'utf8').replaceAll('\r', '')).toBe('192.0.2.1/32\n')
+  })
+  it.each([
+    { deny_cidrs: ['$(touch injected)'] },
+    { allow_cidrs: ['10.0.0.1; flush ruleset'] },
+    { hash: '$(touch injected)' },
+    { default_action: 'accept\nflush ruleset' },
+    { deny_cidrs: ['999.1.1.1/24'] },
+    { deny_cidrs: '10.0.0.0/8' },
+    { port_rules: [{ id: 'x', action: 'open', protocol: 'tcp', port_start: true, src_cidrs: ['0.0.0.0/0'] }] },
+    { port_rules: [{ id: 'x', action: 'open', protocol: 'udp', port_start: 443, port_end: 65536, src_cidrs: [] }] },
+    { port_rules: [{ id: 'x', action: 'execute', protocol: 'tcp', port_start: 1, src_cidrs: [] }] },
+  ])('rejects malformed or unsupported input before emitting files: %j', (patch) => {
+    const { dir, result } = normalize(policy(patch))
+    expect(result.status).not.toBe(0)
+    expect(existsSync(join(dir, 'hash'))).toBe(false)
+    expect(existsSync(join(dir, 'injected'))).toBe(false)
+  })
+  it('renders one nft candidate preserving deny, close, open, implicit-close and Docker scopes', () => {
+    const rule = { protocol: 'tcp', port_start: 443, port_end: 443, src_cidrs: ['192.0.2.0/24'] }
+    const result = render(policy({ port_rules: [{ ...rule, id: 'open', action: 'open' }, { ...rule, id: 'close', action: 'close' }] }))
+    expect(result.status, result.stderr).toBe(0)
+    expect(result.stdout.replaceAll('\r', '').startsWith('delete table inet evofw\nadd table inet evofw\n')).toBe(true)
+    expect(result.stdout).toContain('prerouting fib daddr type local ip saddr @port_src_0_v4 tcp dport 443')
+    expect(result.stdout).toContain('forward ct status dnat ip saddr @port_src_0_v4 tcp dport 443')
+    const input = result.stdout.split('\n').filter((line) => line.startsWith('add rule inet evofw input ')).join('\n')
+    expect(input.indexOf('ip saddr @deny_v4 counter drop')).toBeLessThan(input.indexOf('ip saddr @port_src_1_v4'))
+    expect(input.indexOf('ip saddr @port_src_1_v4')).toBeLessThan(input.indexOf('ip saddr @port_src_0_v4'))
+    expect(input.indexOf('ip saddr @port_src_0_v4')).toBeLessThan(input.indexOf('evofw-port-implicit'))
+    expect(input.indexOf('evofw-port-implicit')).toBeLessThan(input.indexOf('ip saddr @allow_v4'))
+  })
+  it('encodes untrusted rule IDs without injecting nft syntax', () => {
+    const result = render(policy({ port_rules: [{ id: '"; flush ruleset; # $(touch injected)', action: 'close', protocol: 'tcp', port_start: 22, src_cidrs: ['0.0.0.0/0'] }] }))
+    expect(result.status, result.stderr).toBe(0)
+    expect(result.stdout).not.toContain('flush ruleset')
+    expect(result.stdout).not.toContain('touch injected')
+    expect(result.stdout).toMatch(/comment "evofw-port-[a-f0-9]+-(tcp|udp)-v[46]"/)
+  })
+})
+
+describe('agent hardening shell control flow (mock kernel)', () => {
+  it('has valid bash syntax for the agent and installer', () => {
+    for (const name of ['evofw-firewall.sh', 'install.sh']) {
+      const path = fileURLToPath(new URL(`../agent-scripts/${name}`, import.meta.url)).replaceAll('\\', '/')
+      const result = spawnSync(bash, ['--noprofile', '--norc', '-n', path], { encoding: 'utf8' })
+      expect(result.status, result.stderr || String(result.error)).toBe(0)
+    }
+  })
+  it('never mutates nft when every candidate check fails', () => {
+    const { dir, result } = shell(`set -euo pipefail\nRUN_DIR=.; LOG_FILE=log; DENY=(); ALLOW=();\nlog() { :; }\nrender_nft_batch() { echo candidate; }\nnft() { printf '%s\\n' "$*" >> calls; return 1; }\n${fn('apply_nft', '# Collect observed host firewall')}\nif apply_nft; then exit 9; fi\n`)
+    expect(result.status, result.stderr).toBe(0)
+    const calls = readFileSync(join(dir, 'calls'), 'utf8')
+    expect(calls.match(/--check --file/g)).toHaveLength(3)
+    expect(calls.split('\n').filter((line) => line.startsWith('--file'))).toHaveLength(0)
+  })
+  it('commits the exact checked candidate only once', () => {
+    const { dir, result } = shell(`set -euo pipefail\nRUN_DIR=.; LOG_FILE=log; DENY=(10.0.0.0/8); ALLOW=();\nlog() { :; }\nrender_nft_batch() { echo candidate; }\nnft() { printf '%s\\n' "$*" >> calls; return 0; }\n${fn('apply_nft', '# Collect observed host firewall')}\napply_nft\n[[ "$APPLIED" == 1 && "$KERNEL_METHOD" == nft ]]\n`)
+    expect(result.status, result.stderr).toBe(0)
+    const calls = readFileSync(join(dir, 'calls'), 'utf8').split('\n')
+    expect(calls.filter((line) => line.startsWith('--check --file'))).toHaveLength(1)
+    expect(calls.filter((line) => line.startsWith('--file'))).toHaveLength(1)
+  })
+  it('rolls the first ipset back if the second swap fails', () => {
+    const { dir, result } = shell(`set -euo pipefail\nRUN_DIR=.; PORT_RULES_COUNT=0; DEFAULT_ACTION=accept; DENY=(10.0.0.0/8); ALLOW=();\nlog() { :; }\nipset() { printf '%s\\n' "$*" >> calls; if [[ "$1" == save ]]; then echo "create $2 hash:net family inet counters"; elif [[ "$*" == 'swap evofw_allow_stage_v4 evofw_allow_v4' ]]; then return 1; fi; }\niptables() { return 1; }\niptables-save() { printf '%s\\n' '-A INPUT -j EVOFW_INPUT'; }\niptables-restore() { cat >/dev/null; }\n${fn('apply_ipset', 'collect_telemetry() {')}\nif apply_ipset; then exit 9; fi\n`)
+    expect(result.status, result.stderr).toBe(0)
+    const calls = readFileSync(join(dir, 'calls'), 'utf8')
+    expect(calls.match(/swap evofw_deny_stage_v4 evofw_deny_v4/g)).toHaveLength(2)
+    expect(calls.match(/swap evofw_allow_stage_v4 evofw_allow_v4/g)).toHaveLength(1)
+  })
+})
+
+
+describe('agent hardening drift and RouterOS staging', () => {
+  it('ignores ordinary nft counter churn but detects changed enforcement', () => {
+    const dir = workspace()
+    const digest = (packets: number, verdict: string, elements: string[]) => {
+      writeFileSync(join(dir, 'live-kernel.json'), JSON.stringify({ nftables: [
+        { metainfo: { version: '1.0' } },
+        { set: { family: 'inet', table: 'evofw', name: 'deny_port_hits', elem: elements } },
+        { rule: { chain: 'input', handle: packets + 1, expr: [{ counter: { packets, bytes: packets * 100 } }, { [verdict]: null }] } },
+      ] }))
+      const result = spawnSync(python, ['-c', embedded('kernel_digest'), 'nft', dir], { encoding: 'utf8' })
+      expect(result.status, result.stderr).toBe(0)
+      return result.stdout.trim()
+    }
+    expect(digest(1, 'drop', ['a'])).toBe(digest(200, 'drop', ['b']))
+    expect(digest(1, 'drop', [])).not.toBe(digest(1, 'accept', []))
+  })
+  const rscPolicy = (patch = {}): EvaluatedPolicy => ({ hash: 'sha256:' + 'a'.repeat(64), generation: 1, applyVersion: 3, defaultAction: 'accept', policyMode: 'blacklist', denyCidrs: ['10.0.0.0/8'], allowCidrs: ['192.0.2.0/24'], portRules: [], conflictsDropped: 0, syncIntervalSec: 60, chain: [], summary: { sets: 0, rulesDeny: 0, rulesAllow: 0, cidrsDeny: 1, cidrsAllow: 1, overrides: 0, conflictsDropped: 0, portRules: 0 }, ...patch })
+  it('stages both RouterOS address families before canonical-rule activation and retains rollback snapshots', () => {
+    const rsc = renderMikrotikPolicyRsc(rscPolicy())
+    expect(rsc).not.toContain('remove [find list=EVOFW_DENY]')
+    expect(rsc).not.toContain('remove [find list=EVOFW_ALLOW]')
+    expect(rsc.indexOf('address=192.0.2.0/24')).toBeLessThan(rsc.indexOf('/ip firewall filter set [find comment=("evofw-stage-"'))
+    expect(rsc).toContain('disabled=($row->"disabled") comment=($row->"comment")')
+    expect(rsc).toContain('/ip firewall filter remove $new4')
+    expect(rsc).toContain('/ipv6 firewall filter remove $new6')
+    expect(rsc).not.toContain('on-error={}')
+  })
+  it.each([
+    { denyCidrs: ['10.0.0.1; /system reboot'] },
+    { hash: 'a\n/system reboot' },
+    { portRules: [{ id: 'p' }] },
+  ])('rejects invalid or unsupported RouterOS policy: %j', (patch) => {
+    expect(() => renderMikrotikPolicyRsc(rscPolicy(patch))).toThrow()
+  })
+  it.each([
+    ['evofw-deny-drop-input', 'input', 'drop'],
+    ['evofw-deny-drop-forward', 'forward', 'drop'],
+    ['evofw-allow-accept-forward', 'forward', 'accept'],
+    ['evofw-default-drop-forward', 'forward', 'drop'],
+  ])('verifies repaired action/chain and refuses tampered no-op for %s', (comment, chain, action) => {
+    const rsc = renderMikrotikPolicyRsc(rscPolicy())
+    const installer = readFileSync(new URL('../agent-scripts/mikrotik-install.rsc', import.meta.url), 'utf8')
+    expect(rsc).toContain(`filter find comment=${comment}`)
+    expect(rsc).toContain(`get $id chain] != "${chain}"`)
+    expect(rsc).toContain(`get $id action] != "${action}"`)
+    expect(installer).toContain(`filter find comment=${comment}`)
+    expect(installer).toContain(`get $rule chain] != "${chain}"`)
+    expect(installer).toContain(`get $rule action] != "${action}"`)
+    expect(installer.indexOf('owned filter chain/action drift')).toBeLessThan(installer.indexOf('$hash = $EvofwLastHash'))
+  })
+  it('verifies top-of-chain ordering before a RouterOS no-change ACK', () => {
+    const installer = readFileSync(new URL('../agent-scripts/mikrotik-install.rsc', import.meta.url), 'utf8')
+    expect(installer).toContain('$ordered->0')
+    expect(installer).toContain('$ordered->1')
+    expect(installer).toContain('$ordered->2')
+    expect(installer.indexOf('owned filter order drift')).toBeLessThan(installer.indexOf('$hash = $EvofwLastHash'))
+  })
+  it('detects a foreign INPUT ACCEPT preempting the owned ipset jump', () => {
+    const dir = workspace()
+    writeFileSync(join(dir, 'live-ipset.txt'), 'create evofw_deny_v4 hash:net family inet\ncreate evofw_allow_v4 hash:net family inet\n')
+    const digest = (rules: string) => {
+      writeFileSync(join(dir, 'live-iptables.txt'), rules)
+      return spawnSync(python, ['-c', embedded('kernel_digest'), 'ipset', dir], { encoding: 'utf8' })
+    }
+    expect(digest('-A INPUT -j EVOFW_INPUT\n-A INPUT -j ACCEPT\n').status).toBe(0)
+    const preempted = digest('-A INPUT -j ACCEPT\n-A INPUT -j EVOFW_INPUT\n')
+    expect(preempted.status).not.toBe(0)
+    expect(preempted.stderr).toContain('owned INPUT jump is missing or preempted')
+  })
+  it('repositions the owned ipset jump without deleting foreign INPUT rules', () => {
+    const { dir, result } = shell(`set -euo pipefail\nRUN_DIR=.; PORT_RULES_COUNT=0; DEFAULT_ACTION=accept; DENY=(); ALLOW=();\nlog() { :; }\nipset() { if [[ "$1" == save ]]; then echo "create $2 hash:net family inet counters"; fi; }\niptables-save() { printf '%s\\n' '-A INPUT -j ACCEPT' '-A INPUT -j EVOFW_INPUT'; }\niptables-restore() { cat >rules; }\n${fn('apply_ipset', 'collect_telemetry() {')}\napply_ipset\n`)
+    expect(result.status, result.stderr).toBe(0)
+    const rules = readFileSync(join(dir, 'rules'), 'utf8')
+    expect(rules).toContain('-D INPUT -j EVOFW_INPUT')
+    expect(rules).toContain('-I INPUT 1 -j EVOFW_INPUT')
+    expect(rules).not.toContain('-D INPUT -j ACCEPT')
+  })
+  it('keeps RouterOS presence out of packet metrics and retries immutable pending payloads', () => {
+    const installer = readFileSync(new URL('../agent-scripts/mikrotik-install.rsc', import.meta.url), 'utf8')
+    expect(installer).not.toContain('packets\\":1')
+    expect(installer).not.toContain('/v1/agent/heartbeat')
+    expect(installer).toContain('"ip_counters_presence_only"')
+    expect(installer).toContain('"file_commit_not_powerloss_atomic"')
+    expect(installer).toContain('($ack->"report_id") = ($expected->"report_id")')
+    expect(installer).toContain('($ack->"sequence") = ($expected->"sequence")')
+    expect(installer).toContain('http-data=$body output=user as-value duration=1m')
+    expect(installer).toContain('contents=[:serialize $telemetryState to=json')
+  })
+
+  it('updates the RouterOS applied revision only on a structurally verified no-change path', () => {
+    const installer = readFileSync(new URL('../agent-scripts/mikrotik-install.rsc', import.meta.url), 'utf8')
+    const start = installer.indexOf(':if ($hash = $EvofwLastHash && $drift = 0)')
+    const block = installer.slice(start, installer.indexOf('} else={', start))
+    expect(block).toContain(':set EvofwAppliedRevision ($p->"generation")')
+    expect(block).not.toContain('/ip firewall filter set')
+    expect(block).not.toContain('EvofwTrafficEpoch')
+  })
+
+})

apps/api/src/services/analytics-maintenance.test.ts

@@ -0,0 +1,40 @@
+import { describe, expect, it } from 'vitest'
+import { createDb, runMigrations, repos } from '@evofw/db'
+import { mkdtempSync, rmSync, statSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { backfillStatsAnalytics } from './analytics.js'
+
+describe('file-backed analytics maintenance', () => {
+  it('profiles a legacy history with WAL autocheckpoint disabled and preserves all deltas', async () => {
+    const directory = mkdtempSync(join(tmpdir(), 'evofw-analytics-'))
+    const path = join(directory, 'history.db')
+    const { db, sqlite } = createDb(`sqlite:${path}`)
+    try {
+      runMigrations(sqlite)
+      const now = Date.parse('2026-10-26T12:00:00Z'), start = now - 8 * 86400_000
+      const agents = 8, samples = 500
+      for (let i = 0; i < agents; i++) repos.insertAgent(db, { id: `a${i}`, name: `a${i}`, tokenHash: `hash${i}`, tokenPrefix: `t${i}`, status: 'approved' })
+      const insert = sqlite.prepare('INSERT INTO agent_stats_samples(id,agent_id,packets_dropped,packets_accepted,recorded_at) VALUES (?,?,?,?,?)')
+      sqlite.transaction(() => {
+        for (let i = 0; i < samples; i++) for (let a = 0; a < agents; a++) insert.run(`${i}-${a}`, `a${a}`, i * 10, i * 20, new Date(start + i * 60_000).toISOString())
+      })()
+      sqlite.pragma('wal_checkpoint(TRUNCATE)')
+      sqlite.pragma('wal_autocheckpoint = 0')
+      const before = sqlite.prepare('SELECT total_changes() AS n').get() as { n: number }
+      const cpu = process.cpuUsage(), began = performance.now()
+      let processed = 0
+      while (repos.listPendingAnalytics(db, 1).length) processed += await backfillStatsAnalytics(db, new Date(now).toISOString())
+      const elapsedMs = performance.now() - began, used = process.cpuUsage(cpu)
+      const after = sqlite.prepare('SELECT total_changes() AS n').get() as { n: number }
+      expect(processed).toBe(agents * samples)
+      for (const resolution of ['1h', '1d']) {
+        const totals = sqlite.prepare('SELECT sum(packets_dropped) AS dropped,sum(packets_accepted) AS accepted FROM agent_stats_rollups WHERE resolution=?').get(resolution) as { dropped: number; accepted: number }
+        expect(totals.dropped).toBeCloseTo(agents * (samples - 1) * 10)
+        expect(totals.accepted).toBeCloseTo(agents * (samples - 1) * 20)
+      }
+      expect(await backfillStatsAnalytics(db, new Date(now).toISOString())).toBe(0)
+      console.info(JSON.stringify({ samples: agents * samples, elapsedMs, cpuMs: (used.user + used.system) / 1000, walBytes: statSync(`${path}-wal`).size, sqliteChanges: after.n - before.n }))
+    } finally { sqlite.close(); rmSync(directory, { recursive: true, force: true }) }
+  }, 60_000)
+})

apps/api/src/services/analytics.test.ts

@@ -0,0 +1,285 @@
+import { afterEach, describe, expect, it } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { applyReportBodySchema, timeseriesResponseSchema } from '@evofw/shared'
+import { ingestTelemetry } from './telemetry.js'
+import { backfillStatsAnalytics, buildStatsRollups, queryTimeseries, runStatsRetention } from './analytics.js'
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+const base = Date.parse('2026-01-01T00:00:00Z')
+const at = (seconds: number) => new Date(base + seconds * 1000).toISOString()
+afterEach(() => { for (const mem of opened.splice(0)) mem.sqlite.close() })
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  for (const id of ['a', 'b']) repos.insertAgent(mem.db, { id, name: id, tokenPrefix: id, tokenHash: id, status: 'approved' })
+  return mem
+}
+function report(db: ReturnType<typeof memory>['db'], id: string, sequence: number, seconds: number, dropped: number | undefined, accepted = dropped, extra: Record<string, unknown> = {}) {
+  const body = applyReportBodySchema.parse({ telemetry_version: 2, report_id: `${id}-${sequence}`, sequence, boot_id: 'boot',
+    observed_at: at(seconds), interval_ms: 60_000, counter_epoch: { traffic: 't' }, capabilities: ['policy_ipv4'],
+    quality: { partial: false, truncated: false, unsupported: [], lost_reports: 0 },
+    detail_counts: { ip_found: 0, ip_sent: 0, port_found: 0, port_sent: 0 },
+    status: 'ok', packets_dropped: dropped, packets_accepted: accepted, ...extra,
+  })
+  return ingestTelemetry(db, id, body, '127.0.0.1', at(seconds))
+}
+const series = (db: ReturnType<typeof memory>['db'], ids = 'a', seconds = 180) => queryTimeseries(db, { from: at(0), to: at(seconds), resolution: '1m', agent_ids: ids }, at(seconds))
+
+describe('bounded statistics analytics', () => {
+  it('coalesces repeated rollup buckets into one write while preserving counters, flags and lost-report counts', () => {
+    const { db, sqlite } = memory()
+    const before = (sqlite.prepare('SELECT total_changes() AS n').get() as { n: number }).n
+    repos.addStatsRollups(db, [
+      { agentId: 'a', resolution: '1h', bucketStart: at(0), packetsDropped: 10, packetsAccepted: 20, droppedCoverageMs: 1000, acceptedCoverageMs: 1000, partial: 1, unsupportedJson: '["legacy"]', lostReports: 2 },
+      { agentId: 'a', resolution: '1h', bucketStart: at(0), packetsDropped: 30, packetsAccepted: 40, droppedCoverageMs: 2000, acceptedCoverageMs: 3000, estimated: 1, unsupportedJson: '["gap","legacy"]', lostReports: 3 },
+    ])
+    expect((sqlite.prepare('SELECT total_changes() AS n').get() as { n: number }).n - before).toBe(1)
+    const row = sqlite.prepare('SELECT * FROM agent_stats_rollups').get() as Record<string, unknown>
+    expect(row).toMatchObject({ packets_dropped: 40, packets_accepted: 60, dropped_coverage_ms: 3000, accepted_coverage_ms: 4000, partial: 1, estimated: 1, lost_reports: 5 })
+    expect(JSON.parse(String(row.unsupported_json)).sort()).toEqual(['gap', 'legacy'])
+  })
+
+  it('allows only one retention pass per DB while leaving the deferred backlog resumable', async () => {
+    const { db } = memory()
+    for (let i = 0; i < 6; i++) repos.insertStatsSample(db, { id: `flight-${i}`, agentId: 'a', packetsDropped: i * 10, packetsAccepted: 0, recordedAt: at(i * 60) })
+    let progress = 0
+    const running = runStatsRetention(db, at(2 * 86400), 1000, undefined, { maxSamples: 1, onProgress: () => { progress++ } })
+    expect(await runStatsRetention(db, at(2 * 86400))).toBe(0)
+    await running
+    expect(progress).toBe(1)
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(6)
+    await runStatsRetention(db, at(2 * 86400), 1000, undefined, { pauseMs: 0 })
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(0)
+  })
+
+  it('keeps processed predecessors until a bounded backlog fully drains, then prunes without losing its final delta', async () => {
+    const { db, sqlite } = memory()
+    for (let i = 0; i < 4; i++) repos.insertStatsSample(db, { id: `bounded-${i}`, agentId: 'a', packetsDropped: i * 10, packetsAccepted: i * 20, recordedAt: at(i * 60) })
+    const progress: { processed: number; deleted: number; pending: boolean }[] = []
+    const options = { maxSamples: 2, batchSize: 2, pauseMs: 0, onProgress: (value: typeof progress[number]) => progress.push(value) }
+    await runStatsRetention(db, at(2 * 86400), 1000, undefined, options)
+    expect(progress[0]).toMatchObject({ processed: 2, pending: true })
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(4)
+    expect((sqlite.prepare('SELECT count(*) AS n FROM agent_stats_samples WHERE analytics_processed=0').get() as { n: number }).n).toBe(2)
+    await runStatsRetention(db, at(2 * 86400), 1000, undefined, options)
+    expect(progress[1]).toMatchObject({ processed: 2, pending: false })
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(0)
+    const row = sqlite.prepare("SELECT sum(packets_dropped) AS dropped,sum(packets_accepted) AS accepted FROM agent_stats_rollups WHERE resolution='1h'").get()
+    expect(row).toEqual({ dropped: 30, accepted: 60 })
+  })
+
+  it('rolls back an entire claimed backfill batch on write failure and retries exactly once', async () => {
+    const { db, sqlite } = memory()
+    for (let i = 0; i < 3; i++) repos.insertStatsSample(db, { id: `rollback-${i}`, agentId: 'a', packetsDropped: i * 10, packetsAccepted: 0, recordedAt: at(i * 60) })
+    sqlite.exec("CREATE TRIGGER fail_backfill BEFORE INSERT ON agent_stats_rollups BEGIN SELECT RAISE(ABORT, 'backfill failed'); END")
+    await expect(backfillStatsAnalytics(db, at(120))).rejects.toThrow('backfill failed')
+    expect((sqlite.prepare('SELECT sum(analytics_processed) AS n FROM agent_stats_samples').get() as { n: number }).n).toBe(0)
+    sqlite.exec('DROP TRIGGER fail_backfill')
+    expect(await backfillStatsAnalytics(db, at(120))).toBe(3)
+    expect(await backfillStatsAnalytics(db, at(120))).toBe(0)
+    expect(series(db, 'a', 120).items.reduce((sum, row) => sum + (row.packets_dropped ?? 0), 0)).toBe(20)
+  })
+
+  it('aborts real inter-batch sleep promptly, retains pending rows and resumes without double rollups', async () => {
+    const { db } = memory()
+    for (let i = 0; i < 20; i++) repos.insertStatsSample(db, { id: `abort-${i}`, agentId: 'a', packetsDropped: i * 10, packetsAccepted: 0, recordedAt: at(i * 60) })
+    const controller = new AbortController()
+    const timer = setTimeout(() => controller.abort(), 5)
+    const count = await backfillStatsAnalytics(db, at(1200), controller.signal, { batchSize: 2, pauseMs: 500 })
+    clearTimeout(timer)
+    expect(count).toBeGreaterThan(0)
+    expect(count).toBeLessThan(20)
+    const remaining = await backfillStatsAnalytics(db, at(1200), undefined, { pauseMs: 0 })
+    expect(count + remaining).toBe(20)
+    expect(series(db, 'a', 1200).items.reduce((sum, row) => sum + (row.packets_dropped ?? 0), 0)).toBe(190)
+  })
+
+  it('sums contained exact minute intervals into an exact hour and flags only proportional cross-bucket splits', () => {
+    const { db } = memory()
+    report(db, 'a', 1, 0, 0, 0)
+    for (let minute = 1; minute <= 60; minute++) report(db, 'a', minute + 1, minute * 60, minute, minute * 2)
+    const hour = queryTimeseries(db, { from: at(0), to: at(3600), resolution: '1h', agent_ids: 'a' }, at(3600)).items[0]!
+    expect(hour).toMatchObject({ packets_dropped: 60, packets_accepted: 120, coverage: 1, estimated: false, quality: { partial: false } })
+    report(db, 'b', 1, 1800, 0, 0)
+    report(db, 'b', 2, 5400, 60, 0)
+    const split = queryTimeseries(db, { from: at(0), to: at(7200), resolution: '1h', agent_ids: 'b' }, at(7200))
+    expect(split.items.map((point) => point.packets_dropped)).toEqual([30, 30])
+    expect(split.items.every((point) => point.estimated)).toBe(true)
+  })
+
+  it('keeps complete traffic-only reports complete and preserves actual quality reasons at full coverage', () => {
+    const { db } = memory()
+    const trafficOnly = { capabilities: ['policy_ipv4', 'traffic_packets'], detail_counts: undefined }
+    report(db, 'a', 1, 0, 100, 100, trafficOnly)
+    report(db, 'a', 2, 60, 110, 130, trafficOnly)
+    expect(series(db, 'a', 60).items[0]).toMatchObject({ packets_dropped: 10, packets_accepted: 30, coverage: 1, estimated: false, quality: { partial: false, unsupported: [] } })
+    report(db, 'a', 3, 120, 120, 160, { ...trafficOnly, quality: { partial: true, truncated: false, unsupported: ['ip_counter_sampling'], lost_reports: 0 } })
+    const point = series(db, 'a', 120).items[1]!
+    expect(point.coverage).toBe(1)
+    expect(point.quality.unsupported).toContain('ip_counter_sampling')
+    expect(point.quality.unsupported).not.toContain('incomplete_coverage')
+  })
+
+  it('keeps explicit unknown v2 epoch deltas null when backfilling pre013 raw samples', async () => {
+    const { db } = memory()
+    for (const [id, seconds, absolute, delta] of [['v2-1', 0, 100, null], ['v2-2', 60, 1000, null], ['v2-3', 120, 1005, 5]] as const) {
+      repos.insertStatsSample(db, { id, agentId: 'a', telemetryVersion: 2, packetsDropped: absolute, packetsAccepted: 0, deltaDropped: delta, deltaAccepted: delta === null ? null : 0, recordedAt: at(seconds) })
+    }
+    await backfillStatsAnalytics(db, at(120))
+    const response = series(db, 'a', 120)
+    expect(response.items.map((point) => point.packets_dropped)).toEqual([null, 5])
+    expect(response.items[1]!.estimated).toBe(true)
+  })
+
+  it('counts enqueue-interval lost reports once across multiple proportional buckets and immutable retries', () => {
+    const { db } = memory()
+    report(db, 'a', 1, 0, 100)
+    for (const [sequence, seconds, lost] of [[2, 180, 1], [3, 240, 0], [4, 300, 0]] as const) {
+      report(db, 'a', sequence, seconds, 100 + seconds, undefined, { quality: { partial: lost > 0, truncated: false, unsupported: [], lost_reports: lost } })
+    }
+    report(db, 'a', 2, 180, 280, undefined, { quality: { partial: true, truncated: false, unsupported: [], lost_reports: 1 } })
+    const response = series(db, 'a', 300)
+    expect(response.items.map((point) => point.quality.lost_reports)).toEqual([0, 0, 1, 0, 0])
+    expect(response.items.reduce((sum, point) => sum + point.quality.lost_reports, 0)).toBe(1)
+  })
+
+  it('backfills old absolute samples once as estimated before raw retention deletes them', async () => {
+    const { db } = memory()
+    repos.insertStatsSample(db, { id: 'old-1', agentId: 'a', packetsDropped: 100, packetsAccepted: 0, recordedAt: at(0) })
+    repos.insertStatsSample(db, { id: 'old-2', agentId: 'a', packetsDropped: 160, packetsAccepted: 0, recordedAt: at(3600) })
+    expect(await backfillStatsAnalytics(db, at(2 * 86_400))).toBe(2)
+    expect(await backfillStatsAnalytics(db, at(2 * 86_400))).toBe(0)
+    const query = { from: at(0), to: at(3600), resolution: '1h', agent_ids: 'a' }
+    const response = queryTimeseries(db, query, at(2 * 86_400))
+    expect(response.items[0]!.packets_dropped).toBe(60)
+    expect(response.items[0]!.estimated).toBe(true)
+    expect(response.items[0]!.quality.partial).toBe(true)
+    await runStatsRetention(db, at(2 * 86_400))
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(0)
+    expect(queryTimeseries(db, query, at(2 * 86_400))).toEqual(response)
+  })
+
+  it('does not double-account when two yielding backfill runs overlap', async () => {
+    const { db } = memory()
+    for (let i = 0; i <= 30; i++) repos.insertStatsSample(db, { id: `old-${i}`, agentId: 'a', packetsDropped: i * 10, packetsAccepted: 0, recordedAt: at(i * 60) })
+    const counts = await Promise.all([backfillStatsAnalytics(db, at(1800)), backfillStatsAnalytics(db, at(1800))])
+    expect(counts[0]! + counts[1]!).toBe(31)
+    expect(series(db, 'a', 1800).items.reduce((sum, point) => sum + (point.packets_dropped ?? 0), 0)).toBe(300)
+  })
+
+  it('splits each agent observation interval proportionally across UTC buckets and preserves fleet gaps', () => {
+    const { db } = memory()
+    report(db, 'a', 1, 30, 100, 100)
+    report(db, 'a', 2, 150, 220, 160)
+    const a = series(db)
+    expect(a.items.map((p) => p.packets_dropped)).toEqual([30, 60, 30])
+    expect(a.items.map((p) => p.packets_accepted)).toEqual([15, 30, 15])
+    expect(a.items.map((p) => p.coverage)).toEqual([0.5, 1, 0.5])
+    expect(a.items[1]!.pps).toBe(1.5)
+    expect(a.items.every((p) => p.estimated)).toBe(true)
+    expect(timeseriesResponseSchema.safeParse(a).success).toBe(true)
+    const missingB = series(db, 'a,b')
+    expect(missingB.items.every((p) => p.packets_dropped === null && p.pps === null)).toBe(true)
+    expect(missingB.items[1]!.coverage).toBe(0.5)
+    report(db, 'b', 1, 0, 0, 0)
+    report(db, 'b', 2, 180, 180, 0)
+    const fleet = series(db, 'a,b')
+    expect(fleet.items.map((p) => p.packets_dropped)).toEqual([90, 120, 90])
+    expect(fleet.items.map((p) => p.coverage)).toEqual([0.75, 1, 0.75])
+    report(db, 'a', 2, 150, 220, 160)
+    expect(series(db, 'a,b')).toEqual(fleet)
+  })
+
+  it('tracks per-metric baseline times across missing values and skipped sequence intervals', () => {
+    const { db } = memory()
+    report(db, 'a', 1, 0, 100)
+    report(db, 'a', 2, 60, undefined, 110)
+    expect(series(db, 'a', 60).items[0]!.packets_dropped).toBeNull()
+    report(db, 'a', 3, 120, 120, 120)
+    const sample = repos.listStatsSamples(db, 'a').find((row) => row.sequence === 3)!
+    expect(sample.droppedIntervalStart).toBe(at(0))
+    expect(sample.acceptedIntervalStart).toBe(at(60))
+    expect(sample.intervalMs).toBe(120_000)
+    expect(series(db, 'a', 120).items.map((p) => p.packets_dropped)).toEqual([10, 10])
+    const mem = memory()
+    report(mem.db, 'a', 1, 0, 100)
+    report(mem.db, 'a', 4, 180, 130)
+    expect(series(mem.db).items.map((p) => p.packets_dropped)).toEqual([10, 10, 10])
+    expect(series(mem.db).items[2]!.quality.lost_reports).toBe(2)
+  })
+
+  it('leaves initial/new epoch/decreased counter gaps null and represents known zero as zero', () => {
+    const { db } = memory()
+    report(db, 'a', 1, 0, 100)
+    report(db, 'a', 2, 60, 105)
+    report(db, 'a', 3, 120, 1000, 1000, { counter_epoch: { traffic: 'new' } })
+    report(db, 'a', 4, 180, 1000, 1000, { counter_epoch: { traffic: 'new' } })
+    expect(series(db).items.map((p) => p.packets_dropped)).toEqual([5, null, 0])
+    expect(series(db).items[2]!.coverage).toBe(1)
+    expect(series(db).items[2]!.quality.partial).toBe(false)
+  })
+
+  it('rolls back rollups, counter state and sample receipt together on an analytics write error', () => {
+    const { db, sqlite } = memory()
+    report(db, 'a', 1, 0, 100)
+    sqlite.exec("CREATE TRIGGER fail_rollup BEFORE INSERT ON agent_stats_rollups BEGIN SELECT RAISE(ABORT, 'rollup failed'); END;")
+    expect(() => report(db, 'a', 2, 60, 105)).toThrow('rollup failed')
+    expect(repos.getTelemetryState(db, 'a')!.sequence).toBe(1)
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(1)
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(0)
+    sqlite.exec('DROP TRIGGER fail_rollup')
+    report(db, 'a', 2, 60, 105)
+    expect(series(db, 'a', 60).items[0]!.packets_dropped).toBe(5)
+  })
+
+  it('bounds enormous observation spans by retention without compressing their full delta into the retained tail', () => {
+    const quality = { partial: false, truncated: false, unsupported: [], lost_reports: 0 }
+    const now = at(12 * 3600), start = '0001-01-01T00:00:00.000Z'
+    const rows = buildStatsRollups('a', { start, end: now, delta: 100_000 }, { start, end: now, delta: 100_000 }, quality, now, false)
+    expect(rows.length).toBeLessThan(2300)
+    expect(rows.filter((r) => r.resolution === '1d').reduce((sum, r) => sum + (r.packetsDropped ?? 0), 0)).toBeLessThan(20)
+    expect(rows.every((r) => Number.isFinite(r.packetsDropped))).toBe(true)
+    const future = buildStatsRollups('a', { start: at(0), end: at(5000), delta: 100 }, { start: null, end: at(5000), delta: null }, quality, at(60), false)
+    expect(future.every((row) => row.droppedCoverageMs === 0)).toBe(true)
+  })
+
+  it('keeps hourly/daily data after raw deletion, expires every tier and yields between cleanup batches', async () => {
+    const { db, sqlite } = memory()
+    report(db, 'a', 1, 0, 100)
+    report(db, 'a', 2, 7200, 220)
+    repos.upsertIpBlockStats(db, 'a', [{ ip: '8.8.8.8', packets: 10 }], at(0))
+    let yielded = false
+    setImmediate(() => { yielded = true })
+    await runStatsRetention(db, at(2 * 86_400), 1)
+    expect(yielded).toBe(true)
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(0)
+    const hours = queryTimeseries(db, { from: at(0), to: at(7200), resolution: '1h', agent_ids: 'a' }, at(2 * 86_400))
+    expect(hours.items.map((point) => point.packets_dropped)).toEqual([60, 60])
+    repos.resetDisplayedStats(db, 'a')
+    expect(queryTimeseries(db, { from: at(0), to: at(7200), resolution: '1h', agent_ids: 'a' }, at(2 * 86_400))).toEqual(hours)
+    await runStatsRetention(db, at(31 * 86_400))
+    expect(sqlite.prepare("SELECT count(*) AS n FROM agent_stats_rollups WHERE resolution='1h'").get()).toEqual({ n: 0 })
+    expect(repos.listIpBlockStats(db, 'a')).toEqual([])
+    expect(sqlite.prepare("SELECT count(*) AS n FROM agent_stats_rollups WHERE resolution='1d'").get()).not.toEqual({ n: 0 })
+    await runStatsRetention(db, at(91 * 86_400))
+    expect(sqlite.prepare('SELECT count(*) AS n FROM agent_stats_rollups').get()).toEqual({ n: 0 })
+    expect(repos.getTelemetryState(db, 'a')!.sequence).toBe(2)
+  })
+
+  it('validates max1440 after UTC rounding, unknown agents and malformed resolutions', () => {
+    const { db } = memory()
+    expect(queryTimeseries(db, { from: at(0), to: at(1440 * 60), resolution: '1m', agent_ids: 'a' }, at(1440 * 60)).items).toHaveLength(1440)
+    expect(() => queryTimeseries(db, { from: at(1), to: at(1440 * 60 + 1), resolution: '1m' }, at(1440 * 60 + 1))).toThrow(/1440/)
+    expect(() => queryTimeseries(db, { resolution: 'toString' })).toThrow(/resolution/)
+    expect(() => queryTimeseries(db, { agent_ids: 'unknown' })).toThrow(/Agent not found/)
+  })
+
+  it('enforces per-agent IP/port cardinality budgets deterministically', () => {
+    const { db, sqlite } = memory()
+    repos.upsertIpBlockStats(db, 'a', Array.from({ length: 20 }, (_, i) => ({ ip: `192.0.2.${i}`, packets: i + 1 })), at(0))
+    repos.upsertPortBlockStats(db, 'a', Array.from({ length: 20 }, (_, i) => ({ ip: `192.0.2.${i}`, port: 22, protocol: 'tcp', packets: i + 1 })), at(0))
+    repos.enforceDetailBudget(db, 'a', 10, 5)
+    expect(sqlite.prepare("SELECT count(*) AS n FROM agent_ip_block_stats WHERE agent_id='a'").get()).toEqual({ n: 10 })
+    expect(sqlite.prepare("SELECT count(*) AS n FROM agent_port_block_stats WHERE agent_id='a'").get()).toEqual({ n: 5 })
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(20)
+  })
+})

apps/api/src/services/analytics.ts

@@ -0,0 +1,186 @@
+import { repos, type Db, type StatsRollupInsert } from '@evofw/db'
+import { type StatsResolution, type TelemetryQuality, type TimeseriesResponse } from '@evofw/shared'
+import { AppError } from '../plugins/error-handler.js'
+import { setTimeout as sleep } from 'node:timers/promises'
+
+export const RESOLUTION_MS: Record<StatsResolution, number> = { '1m': 60_000, '1h': 3_600_000, '1d': 86_400_000 }
+export const RETENTION_MS: Record<StatsResolution, number> = { '1m': 86_400_000, '1h': 30 * 86_400_000, '1d': 90 * 86_400_000 }
+const floor = (time: number, step: number) => Math.floor(time / step) * step
+const iso = (time: number) => new Date(time).toISOString()
+
+type MetricInterval = { start: string | null; end: string; delta: number | null }
+
+/** Bounded by retention windows, even for a huge offline interval. */
+export function buildStatsRollups(agentId: string, dropped: MetricInterval, accepted: MetricInterval, quality: TelemetryQuality, now: string, legacy: boolean): StatsRollupInsert[] {
+  const nowMs = Date.parse(now)
+  const rows = new Map<string, StatsRollupInsert>()
+  for (const resolution of ['1m', '1h', '1d'] as const) {
+    const step = RESOLUTION_MS[resolution]
+    const earliest = floor(nowMs - RETENTION_MS[resolution], step)
+    const latest = floor(nowMs, step)
+    const row = (bucket: number) => {
+      const key = `${resolution}:${bucket}`
+      let item = rows.get(key)
+      if (!item) {
+        item = { agentId, resolution, bucketStart: iso(bucket), packetsDropped: 0, packetsAccepted: 0,
+          droppedCoverageMs: 0, acceptedCoverageMs: 0, partial: quality.partial ? 1 : 0, truncated: quality.truncated ? 1 : 0, estimated: legacy ? 1 : 0,
+          unsupportedJson: JSON.stringify(quality.unsupported.slice(0, 8)), lostReports: 0 }
+        rows.set(key, item)
+      }
+      return item
+    }
+    for (const [metric, interval] of [['dropped', dropped], ['accepted', accepted]] as const) {
+      const start = interval.start === null ? NaN : Date.parse(interval.start)
+      const end = Date.parse(interval.end)
+      const duration = end - start
+      if (!Number.isFinite(start) || !Number.isFinite(end) || duration <= 0 || interval.delta === null || end > nowMs + 5 * 60_000) continue
+      // Denominator retains the entire observation interval; clipping old buckets
+      // must never compress a long outage's traffic into the retained tail.
+      const first = Math.max(earliest, floor(start, step))
+      const last = Math.min(latest, floor(end - 1, step))
+      for (let bucket = first; bucket <= last; bucket += step) {
+        const overlap = Math.max(0, Math.min(end, bucket + step) - Math.max(start, bucket))
+        if (!overlap) continue
+        const item = row(bucket)
+        const value = interval.delta * (overlap / duration)
+        if (metric === 'dropped') { item.packetsDropped = value; item.droppedCoverageMs = overlap }
+        else { item.packetsAccepted = value; item.acceptedCoverageMs = overlap }
+        if (overlap < duration) item.estimated = 1
+      }
+    }
+    // Preserve quality and unknown-baseline markers without fabricating a zero.
+    const observed = Math.min(nowMs, Math.max(Date.parse(dropped.end), Date.parse(accepted.end)))
+    if (Number.isFinite(observed) && observed >= earliest) {
+      const metadataBucket = Math.min(latest, floor(observed - 1, step))
+      if (metadataBucket >= earliest) row(metadataBucket).lostReports = quality.lost_reports
+    }
+  }
+  return [...rows.values()]
+}
+
+export type TimeseriesQuery = { from?: string; to?: string; resolution?: string; agent_ids?: string }
+
+export function queryTimeseries(db: Db, query: TimeseriesQuery, now = new Date().toISOString()): TimeseriesResponse {
+  const resolution = query.resolution ?? '1h'
+  if (!Object.hasOwn(RESOLUTION_MS, resolution)) throw new AppError('VALIDATION_ERROR', 'resolution must be 1m, 1h or 1d', 400)
+  const selectedResolution = resolution as StatsResolution
+  const step = RESOLUTION_MS[selectedResolution]
+  const endMs = query.to ? Date.parse(query.to) : Date.parse(now)
+  const startMs = query.from ? Date.parse(query.from) : endMs - 86_400_000
+  if (!Number.isFinite(startMs) || !Number.isFinite(endMs) || startMs >= endMs || endMs > Date.parse(now) + 5 * 60_000) throw new AppError('VALIDATION_ERROR', 'Invalid from/to range', 400)
+  const from = floor(startMs, step), to = Math.ceil(endMs / step) * step
+  if ((to - from) / step > 1440) throw new AppError('VALIDATION_ERROR', 'At most 1440 UTC buckets; use a coarser resolution', 400)
+  const allAgents = repos.listAgents(db)
+  const ids = query.agent_ids ? [...new Set(query.agent_ids.split(',').map((id) => id.trim()).filter(Boolean))] : allAgents.filter((agent) => agent.status === 'approved').map((agent) => agent.id)
+  if (ids.length > 100 || ids.some((id) => id.length > 128)) throw new AppError('VALIDATION_ERROR', 'At most 100 agent_ids', 400)
+  if (ids.some((id) => !allAgents.some((agent) => agent.id === id))) throw new AppError('NOT_FOUND', 'Agent not found', 404)
+  const rows = new Map(repos.listStatsRollups(db, ids, selectedResolution, iso(from), iso(to)).map((row) => [row.bucketStart, row]))
+  const items: TimeseriesResponse['items'] = []
+  for (let bucket = from; bucket < to; bucket += step) {
+    const timestamp = iso(bucket), row = rows.get(timestamp)
+    const dropped = row && ids.length && row.droppedAgents === ids.length && row.packetsDropped <= Number.MAX_SAFE_INTEGER ? row.packetsDropped : null
+    const accepted = row && ids.length && row.acceptedAgents === ids.length && row.packetsAccepted <= Number.MAX_SAFE_INTEGER ? row.packetsAccepted : null
+    const coverage = row && ids.length ? Math.min(1, Math.min(row.droppedCoverageMs, row.acceptedCoverageMs) / (step * ids.length)) : 0
+    const partial = !row || !!row.partial || coverage < 1 || dropped === null || accepted === null
+    const unsupported = row?.unsupportedJson ? JSON.parse(row.unsupportedJson) as string[] : []
+    if (coverage < 1 || dropped === null || accepted === null) unsupported.push('incomplete_coverage')
+    else if (partial && unsupported.length === 0) unsupported.push(row?.lostReports ? 'lost_reports' : 'partial_report')
+    items.push({ timestamp, packets_dropped: dropped, packets_accepted: accepted,
+      pps: dropped === null || accepted === null ? null : (dropped + accepted) / (step / 1000),
+      coverage, estimated: !!row?.estimated,
+      quality: { partial, truncated: !!row?.truncated, unsupported: [...new Set(unsupported)], lost_reports: Math.min(Number.MAX_SAFE_INTEGER, row?.lostReports ?? 0) },
+    })
+  }
+  return { from: iso(from), to: iso(to), resolution: selectedResolution, items, agent_ids: ids }
+}
+
+export interface AnalyticsMaintenanceOptions {
+  maxSamples?: number
+  batchSize?: number
+  pauseMs?: number
+  maxDurationMs?: number
+  onProgress?: (value: { processed: number; deleted: number; pending: boolean }) => void
+}
+function maintenanceOptions(options: AnalyticsMaintenanceOptions) {
+  const result = { maxSamples: options.maxSamples ?? 1000, batchSize: options.batchSize ?? 50, pauseMs: options.pauseMs ?? 25, maxDurationMs: options.maxDurationMs ?? 1000 }
+  if (!Number.isInteger(result.maxSamples) || result.maxSamples < 1 || result.maxSamples > 5000 || !Number.isInteger(result.batchSize) || result.batchSize < 1 || result.batchSize > 100 || !Number.isInteger(result.pauseMs) || result.pauseMs < 0 || result.pauseMs > 1000 || !Number.isFinite(result.maxDurationMs) || result.maxDurationMs < 1 || result.maxDurationMs > 10_000) throw new Error('Invalid analytics maintenance budget')
+  return result
+}
+async function pause(ms: number, signal?: AbortSignal) {
+  try { await sleep(ms, undefined, { signal }) } catch (error) { if (!signal?.aborted) throw error }
+}
+const backfills = new WeakSet<Db>()
+const retentions = new WeakSet<Db>()
+
+/** Atomic bounded legacy batches; pending rows remain durable between ticks. */
+export async function backfillStatsAnalytics(db: Db, now = new Date().toISOString(), signal?: AbortSignal, options: AnalyticsMaintenanceOptions = {}) {
+  const budget = maintenanceOptions(options)
+  if (!Number.isFinite(Date.parse(now))) throw new Error('Invalid analytics timestamp')
+  if (backfills.has(db)) return 0
+  backfills.add(db)
+  const started = performance.now()
+  let processed = 0
+  try {
+  for (;;) {
+    if (signal?.aborted || processed >= budget.maxSamples || performance.now() - started >= budget.maxDurationMs) return processed
+    const pending = repos.listPendingAnalytics(db, Math.min(budget.batchSize, budget.maxSamples - processed))
+    if (!pending.length) return processed
+    const claimed = db.transaction((tx) => {
+      let count = 0
+      const combined: StatsRollupInsert[] = []
+      const batchStarted = performance.now()
+      for (const sample of pending) {
+        if (signal?.aborted || (count > 0 && (performance.now() - batchStarted >= 10 || combined.length >= 5000))) break
+        if (!repos.markAnalyticsProcessed(tx, sample.id)) continue
+        const previous = repos.previousStatsSample(tx, sample)
+        const delta = (current: number, before: number | undefined, present: number | null) => present === 0 || before === undefined ? null : current >= before ? current - before : current
+        // An explicit v2 null is an unknown epoch/baseline interval, never an
+        // invitation to infer a delta from unrelated absolute counters.
+        const dropped = sample.droppedPresent === 0 ? null : sample.telemetryVersion === 2 ? sample.deltaDropped : sample.deltaDropped ?? delta(sample.packetsDropped, previous?.packetsDropped, sample.droppedPresent)
+        const accepted = sample.acceptedPresent === 0 ? null : sample.telemetryVersion === 2 ? sample.deltaAccepted : sample.deltaAccepted ?? delta(sample.packetsAccepted, previous?.packetsAccepted, sample.acceptedPresent)
+        const rows = buildStatsRollups(sample.agentId,
+          { start: sample.droppedIntervalStart ?? previous?.recordedAt ?? null, end: sample.recordedAt, delta: dropped },
+          { start: sample.acceptedIntervalStart ?? previous?.recordedAt ?? null, end: sample.recordedAt, delta: accepted },
+          { partial: true, truncated: false, unsupported: ['legacy_estimated_backfill'], lost_reports: 0 }, now, true)
+        combined.push(...rows)
+        count++
+      }
+      repos.addStatsRollups(tx, combined)
+      return count
+    })
+    processed += claimed
+    if (!claimed) return processed
+    await pause(budget.pauseMs, signal)
+  }
+  } finally { backfills.delete(db) }
+}
+
+export async function runStatsRetention(db: Db, now = new Date().toISOString(), batchSize = 1000, signal?: AbortSignal, options: AnalyticsMaintenanceOptions = {}) {
+  if (!Number.isInteger(batchSize) || batchSize < 1 || batchSize > 5000 || !Number.isFinite(Date.parse(now))) throw new Error('Invalid retention parameters')
+  const budget = maintenanceOptions(options)
+  if (retentions.has(db)) return 0
+  retentions.add(db)
+  let deleted = 0
+  try {
+  const processed = await backfillStatsAnalytics(db, now, signal, options)
+  const pending = repos.listPendingAnalytics(db, 1).length > 0 || backfills.has(db)
+  const nowMs = Date.parse(now)
+  const cutoffs = [
+    ['raw', iso(nowMs - 86_400_000)], ['ip', iso(nowMs - 7 * 86_400_000)], ['port', iso(nowMs - 7 * 86_400_000)],
+    ...(['1m', '1h', '1d'] as const).map((resolution) => [resolution, iso(floor(nowMs - RETENTION_MS[resolution], RESOLUTION_MS[resolution]))]),
+  ] as const
+  for (const [kind, cutoff] of cutoffs) {
+    // Pending legacy rows still need their processed raw predecessors.
+    if (kind === 'raw' && pending) continue
+    for (let batch = 0; batch < 2; batch++) {
+      if (signal?.aborted) return deleted
+      const count = repos.deleteAnalyticsBatch(db, kind as 'raw' | 'ip' | 'port' | StatsResolution, cutoff, batchSize)
+      deleted += count
+      if (count < batchSize) break
+      await pause(budget.pauseMs, signal)
+    }
+  }
+  options.onProgress?.({ processed, deleted, pending })
+  return deleted
+  } finally { retentions.delete(db) }
+}

apps/api/src/services/audit-outbox.ts

@@ -0,0 +1,99 @@
+import type { Sqlite } from '@evofw/db'
+import { ingestAuditEventSchema, type IngestAuditEvent } from '@evofw/shared'
+import type { AppConfig } from '../config.js'
+
+export const AUDIT_QUEUE_LIMIT = 10_000
+export const AUDIT_PAYLOAD_BYTES = 16_384
+export const AUDIT_QUARANTINE_LIMIT = 100
+const QUARANTINE_AT = Number.MAX_SAFE_INTEGER
+interface Row { event_id: string; payload_json: string; attempts: number }
+
+/** Repair known legacy field limits, then enforce the actual portal contract. */
+export function normalizePortalAudit(value: unknown): IngestAuditEvent {
+  if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error('Audit payload contract invalid')
+  const event = { ...(value as Record<string, unknown>) }
+  if (typeof event.summary === 'string') event.summary = event.summary.trim().slice(0, 500) || 'Audit event'
+  if (typeof event.action === 'string') event.action = event.action.trim().slice(0, 200)
+  if (event.actor_email !== undefined && !ingestAuditEventSchema.shape.actor_email.safeParse(event.actor_email).success) event.actor_email = null
+  if (event.action === 'settings.update') {
+    const keys = (event.details as Record<string, unknown> | null)?.changed_keys
+    event.details = { changed_keys: Array.isArray(keys) ? keys.filter((key): key is string => typeof key === 'string').slice(0, 100) : [] }
+  }
+  const parsed = ingestAuditEventSchema.safeParse(event)
+  if (!parsed.success) throw new Error('Audit payload contract invalid')
+  return parsed.data
+}
+
+export class AuditOutbox {
+  private running: Promise<void> | null = null
+  private abort = new AbortController()
+  enqueueFailures = 0
+  constructor(private sqlite: Sqlite, private config: AppConfig) {}
+  get enabled() { return !!(this.config.authAuditIngestSecret && this.config.authPortalUrl) }
+  size(): number { return (this.sqlite.prepare('SELECT count(*) AS n FROM audit_outbox').get() as { n: number }).n }
+  enqueue(event: IngestAuditEvent): void {
+    if (!this.enabled) return
+    event = normalizePortalAudit(event)
+    const payload = JSON.stringify(event)
+    if (Buffer.byteLength(payload) > AUDIT_PAYLOAD_BYTES) throw new Error('Audit payload byte limit exceeded')
+    if (this.size() >= AUDIT_QUEUE_LIMIT) throw new Error('Audit outbox capacity reached')
+    this.sqlite.prepare('INSERT INTO audit_outbox(event_id,payload_json,created_at) VALUES (?,?,?)').run(event.event_id, payload, event.created_at)
+  }
+  stats() {
+    const row = this.sqlite.prepare(`SELECT count(*) FILTER(WHERE next_attempt_ms != ?) AS pending,
+      count(*) FILTER(WHERE next_attempt_ms = ?) AS quarantined,
+      min(created_at) FILTER(WHERE next_attempt_ms != ?) AS oldest_at,
+      max(attempts) FILTER(WHERE next_attempt_ms != ?) AS max_attempts FROM audit_outbox`).get(QUARANTINE_AT, QUARANTINE_AT, QUARANTINE_AT, QUARANTINE_AT) as { pending: number; quarantined: number; oldest_at: string | null; max_attempts: number | null }
+    return { ...row, capacity: AUDIT_QUEUE_LIMIT, quarantine_capacity: AUDIT_QUARANTINE_LIMIT, quarantine_recovery: 'Inspect retained local audit events; invalid delivery payloads require operator repair', enqueue_failures: this.enqueueFailures, running: !!this.running, enabled: this.enabled }
+  }
+  flush(now = Date.now()): Promise<void> {
+    if (this.running) return this.running
+    if (this.abort.signal.aborted || !this.enabled) return Promise.resolve()
+    this.running = this.send(now).finally(() => { this.running = null })
+    return this.running
+  }
+  private async send(now: number): Promise<void> {
+    const selected = this.sqlite.prepare('SELECT event_id,payload_json,attempts FROM audit_outbox WHERE next_attempt_ms <= ? ORDER BY next_attempt_ms,created_at LIMIT 25').all(now) as Row[]
+    const rows: Row[] = []
+    const events: IngestAuditEvent[] = []
+    this.sqlite.transaction(() => {
+      for (const row of selected) {
+        try {
+          const event = normalizePortalAudit(JSON.parse(row.payload_json))
+          const payload = JSON.stringify(event)
+          if (event.event_id !== row.event_id || Buffer.byteLength(payload) > AUDIT_PAYLOAD_BYTES) throw new Error('Invalid stored payload')
+          this.sqlite.prepare('UPDATE audit_outbox SET payload_json=? WHERE event_id=?').run(payload, row.event_id)
+          rows.push(row); events.push(event)
+        } catch {
+          this.sqlite.prepare('UPDATE audit_outbox SET next_attempt_ms=?,last_error=? WHERE event_id=?').run(QUARANTINE_AT, 'invalid_portal_contract', row.event_id)
+        }
+      }
+      // Bounded dead letters; original local audit records are never deleted.
+      this.sqlite.prepare('DELETE FROM audit_outbox WHERE event_id IN (SELECT event_id FROM audit_outbox WHERE next_attempt_ms=? ORDER BY created_at DESC,event_id DESC LIMIT -1 OFFSET ?)').run(QUARANTINE_AT, AUDIT_QUARANTINE_LIMIT)
+    })()
+    if (!rows.length) return
+    let failure: string | null = null
+    try {
+      const response = await fetch(`${this.config.authPortalUrl}/api/v1/ingest/audit`, {
+        method: 'POST', redirect: 'error',
+        headers: { Authorization: `Bearer ${this.config.authAuditIngestSecret}`, 'Content-Type': 'application/json' },
+        body: JSON.stringify({ events }),
+        signal: AbortSignal.any([this.abort.signal, AbortSignal.timeout(8000)]),
+      })
+      await response.body?.cancel()
+      if (!response.ok) failure = `HTTP ${response.status}`
+    } catch { failure = this.abort.signal.aborted ? 'shutdown' : 'request failed or timed out' }
+    if (this.abort.signal.aborted) return // Durable rows remain due on next startup.
+    this.sqlite.transaction(() => {
+      for (const row of rows) {
+        if (!failure) this.sqlite.prepare('DELETE FROM audit_outbox WHERE event_id=?').run(row.event_id)
+        else {
+          const attempts = Math.min(row.attempts + 1, 1_000_000)
+          const delay = Math.min(3_600_000, 5000 * 2 ** Math.min(attempts - 1, 10))
+          this.sqlite.prepare('UPDATE audit_outbox SET attempts=?,next_attempt_ms=?,last_error=? WHERE event_id=?').run(attempts, now + delay, failure, row.event_id)
+        }
+      }
+    })()
+  }
+  async close(): Promise<void> { this.abort.abort(); await this.running }
+}

apps/api/src/services/audit.test.ts

@@ -35,7 +35,7 @@ describe('audit API', () => {
     await app.close()
   })
 
-  it('recordAudit pushes to portal when configured', async () => {
+  it('recordAudit durably queues and pushes to portal when the worker flushes', async () => {
     const fetchMock = vi.fn().mockResolvedValue({
       ok: true,
       text: async () => '',
@@ -60,7 +60,11 @@ describe('audit API', () => {
     })
     expect(create.statusCode).toBe(200)
 
+    expect(fetchMock).not.toHaveBeenCalled()
+    expect(app.auditOutbox.size()).toBe(1)
+    await app.auditOutbox.flush()
     expect(fetchMock).toHaveBeenCalledTimes(1)
+    expect(app.auditOutbox.size()).toBe(0)
     const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit]
     expect(url).toBe('http://portal.test/api/v1/ingest/audit')
     expect((init.headers as Record<string, string>).Authorization).toBe(

apps/api/src/services/audit.ts

@@ -1,5 +1,5 @@
 import { randomUUID } from 'node:crypto'
-import type { FastifyBaseLogger, FastifyInstance, FastifyRequest } from 'fastify'
+import type { FastifyInstance, FastifyRequest } from 'fastify'
 import { appendAudit, type AppendAuditInput } from '@evofw/db'
 import type { AuditSeverity, AuditTargetType, IngestAuditEvent } from '@evofw/shared'
 import type { AppConfig } from '../config.js'
@@ -17,11 +17,15 @@ export type RecordAuditInput = {
   ip?: string | null
 }
 
-export function clientIp(request: FastifyRequest): string | null {
+function auditDetails(value: unknown, depth = 0): unknown {
-  const forwarded = request.headers['x-forwarded-for']
+  if (depth > 5) return '[depth limit]'
-  if (typeof forwarded === 'string' && forwarded.trim()) {
+  if (typeof value === 'string') return value.slice(0, 1024)
-    return forwarded.split(',')[0]?.trim() ?? null
+  if (Array.isArray(value)) return value.slice(0, 100).map(item => auditDetails(item, depth + 1))
+  if (value && typeof value === 'object') return Object.fromEntries(Object.entries(value).slice(0, 100).map(([key, item]) => [key, /token|secret|password|authorization|cookie|api_key|enroll_seed/i.test(key) ? '[redacted]' : auditDetails(item, depth + 1)]))
+  return value
 }
+
+export function clientIp(request: FastifyRequest): string | null {
   return request.ip ?? null
 }
 
@@ -46,53 +50,22 @@ export function actorFromRequest(
   }
 }
 
-async function pushAuditToPortal(
-  config: AppConfig,
-  log: FastifyBaseLogger,
-  event: IngestAuditEvent,
-): Promise<void> {
-  const secret = config.authAuditIngestSecret
-  const portalUrl = config.authPortalUrl
-  if (!secret || !portalUrl) return
-
-  const url = `${portalUrl}/api/v1/ingest/audit`
-  const controller = new AbortController()
-  const timeout = setTimeout(() => controller.abort(), 8_000)
-
-  try {
-    const res = await fetch(url, {
-      method: 'POST',
-      headers: {
-        Authorization: `Bearer ${secret}`,
-        'Content-Type': 'application/json',
-        Accept: 'application/json',
-      },
-      body: JSON.stringify({ events: [event] }),
-      signal: controller.signal,
-    })
-    if (!res.ok) {
-      const body = await res.text().catch(() => '')
-      log.warn(
-        { status: res.status, body: body.slice(0, 200), event_id: event.event_id },
-        'audit portal push failed',
-      )
-    }
-  } catch (err) {
-    log.warn({ err, event_id: event.event_id }, 'audit portal push error')
-  } finally {
-    clearTimeout(timeout)
-  }
-}
-
 /**
  * Dual-write audit: local SQLite + auth-portal ingest (source_app fw).
- * Portal push is fire-and-forget; local write is synchronous best-effort.
+ * Local event + bounded durable outbox are persisted together; one worker retries.
  */
 export function recordAudit(
   app: FastifyInstance,
   config: AppConfig,
   input: RecordAuditInput,
 ): void {
+  const sanitized = auditDetails(input.details) as Record<string, unknown> | null
+  const details = sanitized && Buffer.byteLength(JSON.stringify(sanitized)) <= 8192 ? sanitized : sanitized ? { truncated: true } : null
+  input = {
+    ...input, details, summary: input.summary.trim().slice(0, 500) || 'Audit event',
+    actorUserId: input.actorUserId?.slice(0, 256), actorEmail: input.actorEmail?.slice(0, 320),
+    actorName: input.actorName?.slice(0, 256), targetId: input.targetId?.slice(0, 512),
+  }
   const eventId = randomUUID()
   const createdAt = new Date().toISOString()
   const localInput: AppendAuditInput = {
@@ -111,12 +84,6 @@ export function recordAudit(
     createdAt,
   }
 
-  try {
-    appendAudit(app.db, localInput)
-  } catch (err) {
-    app.log.warn({ err, action: input.action }, 'audit_log local append failed')
-  }
-
   const portalEvent: IngestAuditEvent = {
     event_id: eventId,
     source_app: 'fw',
@@ -133,7 +100,17 @@ export function recordAudit(
     created_at: createdAt,
   }
 
-  void pushAuditToPortal(config, app.log, portalEvent)
+  try {
+    app.sqlite.transaction(() => {
+      appendAudit(app.db, localInput)
+      app.auditOutbox.enqueue(portalEvent)
+    })()
+  } catch {
+    app.auditOutbox.enqueueFailures++
+    // Retain the local record even if queue capacity or disk delivery fails.
+    try { appendAudit(app.db, localInput) } catch { /* counted and logged below */ }
+    app.log.error({ action: input.action }, 'audit persistence or delivery enqueue failed')
+  }
 }
 
 export function auditMutation(

apps/api/src/services/backend-hardening.test.ts

@@ -0,0 +1,192 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { mkdtempSync, writeFileSync, rmSync, copyFileSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join, resolve } from 'node:path'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { refreshIpList } from './lists/refresh.js'
+import { addListEntries, rebuildListCascade } from './lists/entries.js'
+import * as hostname from './policy/resolve-hostname.js'
+import { evaluateAgentPolicy } from './policy/evaluate.js'
+import { buildApp } from '../app.js'
+import { loadConfig } from '../config.js'
+import { encryptSecret } from './secret-cipher.js'
+import * as safeFetch from './safe-fetch.js'
+
+const temps: string[] = []
+const dbs: ReturnType<typeof createMemoryDb>[] = []
+// Keep these source/cascade regressions independent of the real socket transport.
+// source-hardening.test.ts separately exercises its DNS and HTTP boundaries.
+beforeEach(() => {
+  vi.spyOn(safeFetch, 'safeFetchJson').mockImplementation(async (url, options) => {
+    const response = await fetch(url.toString(), options)
+    if (!response.ok) throw new Error(`Source HTTP ${response.status}`)
+    return { data: await response.json(), bytes: 1 }
+  })
+})
+afterEach(() => {
+  vi.unstubAllGlobals()
+  vi.unstubAllEnvs()
+  vi.restoreAllMocks()
+  for (const mem of dbs.splice(0)) mem.sqlite.close()
+  for (const dir of temps.splice(0)) rmSync(dir, { recursive: true })
+})
+function memory() {
+  const mem = createMemoryDb(); dbs.push(mem); runMigrations(mem.sqlite); return mem
+}
+function agent(db: ReturnType<typeof memory>['db'], id: string) {
+  repos.insertAgent(db, { id, name: id, tokenPrefix: id, tokenHash: id, status: 'approved' })
+}
+function list(db: ReturnType<typeof memory>['db'], id: string, type = 'json_url') {
+  repos.insertIpList(db, { id, name: id, type, configJson: JSON.stringify({ url: 'https://feed.test/prefixes' }) })
+}
+
+describe('backend data integrity', () => {
+  it.each(['static', 'json_url'])('atomically publishes a %s child and DNS-dependent ancestors after failure/recovery', async (type) => {
+    const { db } = memory(); agent(db, 'a'); repos.ensureSharedSetAssigned(db, 'a')
+    list(db, 'child', type); list(db, 'parent', 'static')
+    if (type === 'static') repos.updateIpList(db, 'child', { configJson: JSON.stringify({ items: [{ kind: 'ip', value: '1.1.1.1' }] }) })
+    repos.replaceIpListEntries(db, 'child', ['1.1.1.1'])
+    repos.updateIpList(db, 'parent', { configJson: JSON.stringify({ items: [{ kind: 'list', value: 'child' }, { kind: 'hostname', value: 'parent.test' }] }) })
+    repos.insertPolicyRule(db, { id: 'r', setId: 'set-shared-default', priority: 10, action: 'deny', listId: 'parent' })
+    const dns = vi.spyOn(hostname, 'resolveHostnameToCidrs').mockResolvedValue(['9.9.9.9/32'])
+    await rebuildListCascade(db, 'child')
+    const beforeChild = repos.getIpList(db, 'child')!
+    const beforeParent = repos.getIpList(db, 'parent')!
+    const generation = repos.getAgent(db, 'a')!.policyGeneration
+    dns.mockRejectedValueOnce(new Error('parent DNS unavailable'))
+    if (type === 'static') await expect(addListEntries(db, 'child', { values: ['2.2.2.2'] })).rejects.toThrow('parent DNS unavailable')
+    else {
+      vi.stubGlobal('fetch', vi.fn().mockImplementation(() => Promise.resolve(Response.json(['2.2.2.2']))))
+      await refreshIpList(db, 'child')
+      expect(repos.getIpList(db, 'child')!.lastError).toContain('parent DNS unavailable')
+    }
+    expect(repos.getIpList(db, 'child')!.configJson).toBe(beforeChild.configJson)
+    expect(repos.getIpList(db, 'parent')!.configJson).toBe(beforeParent.configJson)
+    expect(repos.listIpListEntries(db, 'child').map((row) => row.cidr)).toEqual(['1.1.1.1/32'])
+    expect(repos.listIpListEntries(db, 'parent').map((row) => row.cidr).sort()).toEqual(['1.1.1.1/32', '9.9.9.9/32'])
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(generation)
+    if (type === 'static') await addListEntries(db, 'child', { values: ['2.2.2.2'] })
+    else await refreshIpList(db, 'child')
+    expect(repos.listIpListEntries(db, 'parent').map((row) => row.cidr)).toContain('2.2.2.2/32')
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBeGreaterThan(generation)
+    expect(repos.getIpList(db, 'child')!.lastError).toBeNull()
+  })
+
+  it('decrypts autocomplete credentials and validates manual config entries at creation', async () => {
+    vi.stubEnv('EVOFW_SECRET_KEY', 'k'.repeat(32))
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } })
+    try {
+      repos.setSetting(app.db, 'evobgp_api_url', 'https://bgp.test')
+      repos.setSetting(app.db, 'evobgp_api_token', encryptSecret('upstream-secret'))
+      const fetcher = vi.fn().mockResolvedValue(Response.json({ items: [{ id: 'one', community: '65000:1' }], has_more: false }))
+      vi.stubGlobal('fetch', fetcher)
+      const autocomplete = await app.inject({ method: 'GET', url: '/api/v1/integrations/evobgp/communities' })
+      expect(autocomplete.statusCode).toBe(200)
+      expect(fetcher.mock.calls[0]![1].headers.Authorization).toBe('Bearer upstream-secret')
+      expect(autocomplete.body).not.toContain('upstream-secret')
+      const invalid = await app.inject({ method: 'POST', url: '/api/v1/lists', payload: { name: 'invalid', type: 'static', config: { items: [{ kind: 'cidr', value: '999.1.1.1/24' }] } } })
+      expect(invalid.statusCode).toBe(400)
+      expect(repos.listIpLists(app.db)).toHaveLength(0)
+      const created = await app.inject({ method: 'POST', url: '/api/v1/lists', payload: { name: 'valid', type: 'static', config: { items: [{ kind: 'cidr', value: '192.0.2.199/24' }] } } })
+      expect(created.statusCode).toBe(200)
+      expect(repos.listIpListEntries(app.db, created.json().id).map((row) => row.cidr)).toEqual(['192.0.2.0/24'])
+    } finally { await app.close() }
+  })
+
+  it('rolls back failed DDL and migration stamp, then retries cleanly', () => {
+    const mem = createMemoryDb(); dbs.push(mem)
+    const dir = mkdtempSync(join(tmpdir(), 'evofw-migrations-')); temps.push(dir)
+    const file = join(dir, '001_test.sql')
+    writeFileSync(file, 'CREATE TABLE example(id TEXT PRIMARY KEY); INSERT INTO nonexistent VALUES (1);')
+    expect(() => runMigrations(mem.sqlite, dir)).toThrow()
+    expect(mem.sqlite.prepare("SELECT name FROM sqlite_master WHERE name='example'").get()).toBeUndefined()
+    expect(mem.sqlite.prepare('SELECT count(*) AS n FROM _migrations').get()).toEqual({ n: 0 })
+    expect(mem.sqlite.pragma('foreign_keys', { simple: true })).toBe(1)
+    writeFileSync(file, 'CREATE TABLE example(id TEXT PRIMARY KEY);')
+    runMigrations(mem.sqlite, dir)
+    expect(mem.sqlite.prepare('SELECT count(*) AS n FROM _migrations').get()).toEqual({ n: 1 })
+  })
+
+  it('preserves child rows through populated agents table rebuilds', () => {
+    const mem = createMemoryDb(); dbs.push(mem)
+    const dir = mkdtempSync(join(tmpdir(), 'evofw-upgrade-')); temps.push(dir)
+    const source = resolve('../../packages/db/migrations')
+    for (const name of ['001_initial.sql', '002_policy_sets.sql', '003_install_links.sql']) copyFileSync(join(source, name), join(dir, name))
+    runMigrations(mem.sqlite, dir)
+    mem.sqlite.prepare("INSERT INTO agents(id,name,token_prefix,token_hash,status) VALUES('a','a','a','a','approved')").run()
+    mem.sqlite.prepare("INSERT INTO agent_policy_sets(agent_id,set_id,sort) VALUES('a','set-shared-default',0)").run()
+    mem.sqlite.prepare("INSERT INTO ip_overrides(id,agent_id,cidr,action) VALUES('o','a','1.1.1.1/32','deny')").run()
+    runMigrations(mem.sqlite)
+    expect(repos.listSetsForAgent(mem.db, 'a')).toHaveLength(1)
+    expect(repos.listOverrides(mem.db, 'a')).toHaveLength(1)
+    expect(mem.sqlite.pragma('foreign_key_check')).toEqual([])
+    expect(mem.sqlite.pragma('foreign_keys', { simple: true })).toBe(1)
+  })
+
+  it('replaces 50k entries in bounded SQL batches and skips identical snapshots', () => {
+    const { db } = memory(); list(db, 'large')
+    const cidrs = Array.from({ length: 50_000 }, (_, i) => `10.${i >> 8}.${i & 255}.1/24`)
+    expect(repos.replaceIpListEntries(db, 'large', cidrs)).toBe(true)
+    expect(repos.listIpListEntries(db, 'large')).toHaveLength(50_000)
+    expect(repos.replaceIpListEntries(db, 'large', cidrs.reverse())).toBe(false)
+  })
+
+  it('fetches every cursor page, retains last good on a later-page failure and skips unchanged writes', async () => {
+    const { db } = memory(); agent(db, 'a'); list(db, 'l', 'evobgp_community')
+    repos.updateIpList(db, 'l', { configJson: JSON.stringify({ api_url: 'https://bgp.test', api_token: 'token', community_id: '01234567-0123-0123-0123-012345678901' }) })
+    repos.insertPolicyRule(db, { id: 'r', setId: 'set-shared-default', priority: 10, action: 'deny', listId: 'l' })
+    repos.ensureSharedSetAssigned(db, 'a')
+    const fetcher = vi.fn().mockResolvedValueOnce(Response.json({ prefixes: ['192.0.2.99/24'], has_more: true, next_cursor: 'page2' })).mockResolvedValueOnce(Response.json({ prefixes: ['2001:db8::1/32'], has_more: false }))
+    vi.stubGlobal('fetch', fetcher)
+    await refreshIpList(db, 'l')
+    expect(fetcher.mock.calls[1]![0]).toContain('cursor=page2')
+    expect(repos.listIpListEntries(db, 'l').map((r) => r.cidr).sort()).toEqual(['192.0.2.0/24', '2001:db8::/32'])
+    const generation = repos.getAgent(db, 'a')!.policyGeneration
+    const rowIds = repos.listIpListEntries(db, 'l').map((r) => r.id)
+    fetcher.mockResolvedValueOnce(Response.json({ prefixes: ['2001:db8::/32', '192.0.2.0/24'], has_more: false }))
+    await refreshIpList(db, 'l')
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(generation)
+    expect(repos.listIpListEntries(db, 'l').map((r) => r.id)).toEqual(rowIds)
+    fetcher.mockResolvedValueOnce(Response.json({ prefixes: ['8.8.8.8'], has_more: true, next_cursor: 'bad' })).mockRejectedValueOnce(new Error('upstream unavailable'))
+    await refreshIpList(db, 'l')
+    expect(repos.getIpList(db, 'l')!.lastError).toContain('unavailable')
+    expect(repos.listIpListEntries(db, 'l').map((r) => r.id)).toEqual(rowIds)
+  })
+
+  it('rejects malformed feed without clearing last good entries', async () => {
+    const { db } = memory(); list(db, 'l'); repos.replaceIpListEntries(db, 'l', ['8.8.8.8'])
+    for (const body of [{ error: 'bad' }, ['invalid.prefix/30'], [null]]) {
+      vi.stubGlobal('fetch', vi.fn().mockResolvedValue(Response.json(body)))
+      await refreshIpList(db, 'l')
+      expect(repos.getIpList(db, 'l')!.lastError).toBeTruthy()
+      expect(repos.listIpListEntries(db, 'l').map((r) => r.cidr)).toEqual(['8.8.8.8/32'])
+    }
+  })
+
+  it('prevents self clone and cross-agent override deletion and rejects duplicate reorder', () => {
+    const { db } = memory(); agent(db, 'a'); agent(db, 'b')
+    repos.insertOverride(db, { id: 'o', agentId: 'a', cidr: '8.8.8.8/32', action: 'deny' })
+    expect(() => repos.cloneRulesFrom(db, 'a', 'a', true)).toThrow(/itself/)
+    expect(repos.deleteOverride(db, 'o', 'b').changes).toBe(0)
+    expect(repos.listOverrides(db, 'a')).toHaveLength(1)
+    for (const [id, priority] of [['r1', 9000], ['r2', 9010]] as const) repos.insertPolicyRule(db, { id, setId: 'set-shared-default', priority, action: 'deny', cidr: '8.8.8.8/32' })
+    expect(() => repos.reorderPolicyRules(db, 'set-shared-default', ['r1', 'r1'])).toThrow(/exactly once/)
+    repos.reorderPolicyRules(db, 'set-shared-default', ['r2', 'r1'])
+    expect(repos.listPolicyRules(db, 'set-shared-default').map((r) => r.id)).toEqual(['r2', 'r1'])
+  })
+
+  it('hash depends on policy content and unchanged manual refresh does not bump agents', async () => {
+    const { db } = memory(); agent(db, 'a'); repos.ensureSharedSetAssigned(db, 'a')
+    list(db, 'manual', 'static')
+    repos.updateIpList(db, 'manual', { configJson: JSON.stringify({ items: [{ kind: 'ip', value: '8.8.8.8' }] }) })
+    repos.insertPolicyRule(db, { id: 'r', setId: 'set-shared-default', priority: 10, action: 'deny', listId: 'manual' })
+    await rebuildListCascade(db, 'manual')
+    const generation = repos.getAgent(db, 'a')!.policyGeneration
+    const hash = evaluateAgentPolicy(db, 'a').hash
+    await refreshIpList(db, 'manual')
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(generation)
+    repos.bumpAgentGeneration(db, 'a')
+    expect(evaluateAgentPolicy(db, 'a').hash).toBe(hash)
+    expect(() => repos.deleteIpList(db, 'manual')).toThrow(/referenced/)
+  })
+})

apps/api/src/services/detail-stats.test.ts

@@ -0,0 +1,99 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+afterEach(() => { vi.restoreAllMocks(); for (const mem of opened.splice(0)) mem.sqlite.close() })
+const first = '2026-01-01T00:00:00.000Z'
+const second = '2026-01-01T00:01:00.000Z'
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  for (const id of ['a', 'b']) repos.insertAgent(mem.db, { id, name: id, tokenPrefix: id, tokenHash: id, status: 'approved' })
+  return mem
+}
+function hits(count: number, packets: number) {
+  return Array.from({ length: count }, (_, index) => ({ ip: `10.${Math.floor(index / 256)}.${index % 256}.1`, port: index + 1, protocol: 'tcp', packets }))
+}
+
+describe('bounded detail counter batches', () => {
+  for (const count of [200, 500]) it(`preserves ${count} detail baselines and prepares bounded queries`, () => {
+    const { db, sqlite } = memory()
+    // Older rows sharing the same IP must not expand the port prefetch result.
+    repos.upsertPortBlockStats(db, 'a', Array.from({ length: 900 }, (_, index) => ({ ip: '10.0.0.1', port: index + 1000, protocol: 'udp', packets: 1 })), first)
+    repos.upsertIpBlockStats(db, 'b', hits(count, 999), first)
+    const prepare = vi.spyOn(sqlite, 'prepare')
+    const started = performance.now()
+    repos.upsertIpBlockStats(db, 'a', hits(count, 100), first, { counterEpoch: 'one' })
+    repos.upsertPortBlockStats(db, 'a', hits(count, 100), first, { counterEpoch: 'one' })
+    const calls = prepare.mock.calls.map(([query]) => String(query))
+    expect(calls.filter((query) => /^select /i.test(query))).toHaveLength(2 * Math.ceil(count / 200))
+    expect(calls.filter((query) => /^insert into /i.test(query))).toHaveLength(2)
+    expect(calls.filter((query) => /^select /i.test(query)).every((query) => query.includes(' in (') || query.includes('"port" = ?'))).toBe(true)
+    prepare.mockRestore()
+    const unknown = vi.fn()
+    repos.upsertIpBlockStats(db, 'a', hits(count, 105), second, { counterEpoch: 'one', onUnknownInterval: unknown })
+    repos.upsertPortBlockStats(db, 'a', hits(count, 105), second, { counterEpoch: 'one', onUnknownInterval: unknown })
+    expect(unknown).not.toHaveBeenCalled()
+    expect(repos.listIpBlockStats(db, 'a', count + 1)).toHaveLength(count)
+    expect(repos.listIpBlockStats(db, 'a', count + 1).every((row) => row.packets === 5 && row.firstSeenAt === first && row.lastSeenAt === second)).toBe(true)
+    expect(sqlite.prepare('SELECT count(*) AS n FROM agent_port_block_stats WHERE agent_id = ? AND protocol = ? AND packets = ?').get('a', 'tcp', 5)).toEqual({ n: count })
+    expect(repos.listIpBlockStats(db, 'b', count + 1).every((row) => row.packets === 999)).toBe(true)
+    repos.upsertIpBlockStats(db, 'a', hits(count, 1), second, { counterEpoch: 'two', onUnknownInterval: unknown })
+    repos.upsertPortBlockStats(db, 'a', hits(count, 1), second, { counterEpoch: 'two', onUnknownInterval: unknown })
+    expect(unknown).toHaveBeenCalledTimes(2 * count)
+    expect(repos.listIpBlockStats(db, 'a', count).every((row) => row.packets === 5)).toBe(true)
+    expect(sqlite.prepare('SELECT count(*) AS n FROM agent_port_block_stats WHERE agent_id = ? AND protocol = ? AND packets = ?').get('a', 'tcp', 5)).toEqual({ n: count })
+    console.info(`detail batch ${count}: baseline + delta + epoch reset, both kinds ${(performance.now() - started).toFixed(1)}ms`)
+  })
+
+  it('preserves sequential duplicates, legacy resets, timestamps and presence re-hits', () => {
+    const { db } = memory()
+    const ip = '192.0.2.1'
+    const duplicate = [10, 15, 2, 4].map((packets) => ({ ip, packets, port: 22, protocol: 'tcp' }))
+    repos.upsertIpBlockStats(db, 'a', duplicate, first)
+    repos.upsertPortBlockStats(db, 'a', duplicate, first)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(19)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(19)
+    repos.upsertIpBlockStats(db, 'a', [{ ip, packets: 4 }], second)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.lastSeenAt).toBe(first)
+    repos.resetIpBlockStatsBaselines(db, 'a')
+    repos.resetPortBlockStatsBaselines(db, 'a')
+    repos.upsertIpBlockStats(db, 'a', [{ ip, packets: 3 }], second)
+    repos.upsertPortBlockStats(db, 'a', [{ ip, port: 22, protocol: 'TCP', packets: 3 }], second)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(22)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(22)
+    const presence = [{ ip: '192.0.2.2', packets: 50 }, { ip: '192.0.2.2', packets: 99 }]
+    repos.upsertIpBlockStats(db, 'a', presence, first, { mode: 'presence' })
+    repos.upsertIpBlockStats(db, 'a', presence, second, { mode: 'presence' })
+    expect(repos.listIpBlockStats(db, 'a').find((row) => row.ip === '192.0.2.2')!.packets).toBe(1)
+    repos.upsertIpBlockStats(db, 'a', presence, '2026-01-01T00:04:00.000Z', { mode: 'presence' })
+    expect(repos.listIpBlockStats(db, 'a').find((row) => row.ip === '192.0.2.2')!.packets).toBe(2)
+  })
+
+  it('rolls back the complete batch on overflow and preserves callback semantics', () => {
+    const { db, sqlite } = memory()
+    repos.upsertIpBlockStats(db, 'a', [{ ip: '192.0.2.2', packets: Number.MAX_SAFE_INTEGER }], first)
+    const batch = [{ ip: '192.0.2.1', packets: 1 }, { ip: '192.0.2.2', packets: 1 }]
+    expect(() => repos.upsertIpBlockStats(db, 'a', batch, second)).toThrow('safe integer')
+    expect(sqlite.prepare('SELECT count(*) AS n FROM agent_ip_block_stats').get()).toEqual({ n: 1 })
+    const overflow = vi.fn()
+    repos.upsertIpBlockStats(db, 'a', batch, second, { onOverflow: overflow })
+    expect(overflow).toHaveBeenCalledOnce()
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(Number.MAX_SAFE_INTEGER)
+  })
+
+  it('keeps duplicate epoch resets unknown and unchanged port timestamps stable', () => {
+    const { db, sqlite } = memory()
+    const unknown = vi.fn()
+    const rows = [100, 105, 1, 4].map((packets) => ({ ip: '192.0.2.1', port: 22, protocol: 'tcp', packets }))
+    repos.upsertIpBlockStats(db, 'a', rows, first, { counterEpoch: 'one', onUnknownInterval: unknown })
+    repos.upsertPortBlockStats(db, 'a', rows, first, { counterEpoch: 'one', onUnknownInterval: unknown })
+    expect(unknown).toHaveBeenCalledTimes(4)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(8)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(8)
+    repos.upsertPortBlockStats(db, 'a', [rows[3]!], second, { counterEpoch: 'one' })
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.lastSeenAt).toBe(first)
+    sqlite.prepare('UPDATE agent_port_block_stats SET packets = ?').run(Number.MAX_SAFE_INTEGER)
+    expect(() => repos.upsertPortBlockStats(db, 'a', [{ ...rows[3]!, packets: 5 }], second, { counterEpoch: 'one' })).toThrow('safe integer')
+    expect(sqlite.prepare('SELECT last_reported_packets AS n FROM agent_port_block_stats').get()).toEqual({ n: 4 })
+  })
+})

apps/api/src/services/enrollment.test.ts

@@ -0,0 +1,108 @@
+import { afterAll, describe, expect, it } from 'vitest'
+import { repos } from '@evofw/db'
+import { buildApp } from '../app.js'
+import type { AppConfig } from '../config.js'
+import { linkEnrollmentCredential } from './enrollment.js'
+
+const config: AppConfig = { databaseUrl: 'sqlite::memory:', jwtSecret: 'test-secret', jwtTtlHours: 24, serverPort: 8080, staticDir: null, logLevel: 'error', authRequired: false, authIssuer: 'https://auth.test', authPortalUrl: 'http://localhost:5175', publicBaseUrl: 'https://fw.example.com', enrollSeed: 'private-server-key', corsOrigins: [], authAuditIngestSecret: null, secretKey: null, statsRetentionDays: 30 }
+const token = (label: string) => 'evofw_' + label.padEnd(43, 'a')
+describe('one-time enrollment and recoverable credential rotation', () => {
+  const appPromise = buildApp({ memory: true, config })
+  afterAll(async () => (await appPromise).close())
+  async function invite(name: string, platform = 'linux') {
+    const app = await appPromise
+    const response = await app.inject({ method: 'POST', url: '/api/v1/install-links', payload: { name, platform } })
+    expect(response.statusCode, response.body).toBe(201)
+    const link = response.json()
+    const credential = linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, config.enrollSeed)
+    const body = { name, platform, token: token(name), install_link_id: link.id, client_version: platform === 'linux' ? 'linux/10' : 'rsc/10' }
+    const enroll = (changes = {}, presented = credential) => app.inject({ method: 'POST', url: '/v1/agent/enroll', headers: { 'x-evofw-enrollment-credential': presented }, payload: { ...body, ...changes } })
+    return { app, link, body, enroll, credential }
+  }
+  it('never renders a reusable global credential and consumes only successful enrollments', async () => {
+    const { app, link, enroll, credential } = await invite('scoped')
+    for (const path of ['/agent-install/' + link.id, '/' + link.slug]) {
+      const script = await app.inject({ url: path })
+      expect(script.statusCode).toBe(200)
+      expect(script.headers['cache-control']).toBe('no-store')
+      expect(script.body).toContain(credential)
+      expect(script.body).not.toContain(config.enrollSeed)
+      expect(script.body).not.toContain('export EVOFW_SEED=')
+    }
+    expect((await enroll({}, 'wrong')).statusCode).toBe(401)
+    expect((await enroll({ token: 'bad' })).statusCode).toBe(400)
+    expect(repos.getInstallLink(app.db, link.id)?.consumedAt).toBeNull()
+    const first = await enroll()
+    expect(first.statusCode, first.body).toBe(201)
+    expect((await enroll()).statusCode).toBe(200)
+    expect((await enroll({ token: token('different') })).statusCode).toBe(409)
+    expect(repos.getInstallLink(app.db, link.id)?.consumedAt).toBeTruthy()
+    const listing = await app.inject({ url: '/api/v1/install-links' })
+    expect(listing.body).not.toContain(credential)
+    expect(listing.body).not.toContain('token_hash')
+  })
+  it('serializes competing consumption and never overwrites the enrolled token', async () => {
+    const { app, body, link, enroll } = await invite('race')
+    const results = await Promise.all([enroll(), enroll({ token: token('race-other') })])
+    expect(results.map(x => x.statusCode).sort()).toEqual([201, 409])
+    const agent = repos.getAgent(app.db, link.agent_id)!
+    expect(agent.tokenPrefix).toBe(body.token.slice(0, 12))
+  })
+  it.each(['revoke', 'expire', 'agent-revoke'])('rejects %s even for an already downloaded script and exact consumed retry', async (kind) => {
+    const { app, link, enroll } = await invite('invalid-' + kind)
+    expect((await enroll()).statusCode).toBe(201)
+    if (kind === 'revoke') repos.revokeInstallLink(app.db, link.id)
+    if (kind === 'expire') app.sqlite.prepare('UPDATE agent_install_links SET expires_at=? WHERE id=?').run('2000-01-01T00:00:00.000Z', link.id)
+    if (kind === 'agent-revoke') repos.updateAgent(app.db, link.agent_id, { status: 'revoked' })
+    expect((await enroll()).statusCode).toBe(410)
+  })
+  it('refuses legacy server seed enrollment', async () => {
+    const app = await appPromise
+    expect((await app.inject({ method: 'POST', url: '/v1/agent/enroll', headers: { 'x-evofw-seed': config.enrollSeed }, payload: { name: 'legacy', token: token('legacy') } })).statusCode).toBeGreaterThanOrEqual(400)
+  })
+  it('gates old clients, exchanges atomically, and permits only exact bounded old-token replay', async () => {
+    const { app, link, body, enroll } = await invite('rotate')
+    expect((await enroll()).statusCode).toBe(201)
+    repos.updateAgent(app.db, link.agent_id, { status: 'approved', clientVersion: 'linux/8' })
+    const request = () => app.inject({ method: 'POST', url: `/api/v1/agents/${link.agent_id}/token-rotation` })
+    expect((await request()).statusCode).toBe(409)
+    const unsupported = await app.inject({ url: `/api/v1/agents/${link.agent_id}/token-rotation` })
+    expect(unsupported.json()).toMatchObject({ agent_id: link.agent_id, supported: false, client_version: 'linux/8', rotation: null })
+    expect(unsupported.body).not.toContain('token_hash')
+    expect(unsupported.body).not.toContain(body.token)
+    repos.updateAgent(app.db, link.agent_id, { clientVersion: 'linux/10' })
+    const requested = await request()
+    expect(requested.statusCode, requested.body).toBe(202)
+    const { request_id } = requested.json()
+    expect((await request()).json().request_id).toBe(request_id)
+    const oldHeaders = { authorization: 'Bearer ' + body.token }
+    expect((await app.inject({ url: '/v1/agent/credentials', headers: oldHeaders })).json().rotation.request_id).toBe(request_id)
+    const payload = { request_id, next_token: token('next') }
+    const exchange = (changes = {}, auth = body.token) => app.inject({ method: 'POST', url: '/v1/agent/token-rotation', headers: { authorization: 'Bearer ' + auth }, payload: { ...payload, ...changes } })
+    const ack = await exchange()
+    expect(ack.statusCode, ack.body).toBe(200)
+    expect((await exchange()).json()).toEqual(ack.json()) // ACK lost: persisted pending body retries unchanged.
+    expect((await exchange({}, payload.next_token)).statusCode).toBe(200) // Config commit succeeded, pending cleanup failed.
+    expect((await exchange({ next_token: token('unrelated') })).statusCode).toBe(401)
+    for (const path of ['/v1/agent/policy', '/v1/agent/policy.rsc', '/v1/agent/credentials']) expect((await app.inject({ url: path, headers: oldHeaders })).statusCode).toBe(401)
+    expect((await app.inject({ url: '/v1/agent/policy', headers: { authorization: 'Bearer ' + payload.next_token } })).statusCode).toBe(200)
+    expect((await request()).statusCode).toBe(409)
+    app.sqlite.prepare('UPDATE agent_token_rotations SET expires_at=? WHERE agent_id=?').run('2000-01-01T00:00:00.000Z', link.agent_id)
+    expect((await exchange()).statusCode).toBe(401)
+    const proof = await app.inject({ url: '/v1/agent/credentials', headers: { authorization: 'Bearer ' + payload.next_token } })
+    expect(proof.json()).toMatchObject({ client_id: link.agent_id, completed_rotation: { request_id } })
+    // A newer operator request must not erase the previous completed receipt.
+    expect((await request()).statusCode).toBe(202)
+    expect((await app.inject({ url: '/v1/agent/credentials', headers: { authorization: 'Bearer ' + payload.next_token } })).json().completed_rotation).toEqual({ request_id })
+  })
+  it('never revives a revoked agent through a completed exchange', async () => {
+    const { app, link, body, enroll } = await invite('revoke-exchange')
+    await enroll(); repos.updateAgent(app.db, link.agent_id, { status: 'approved' })
+    const requested = await app.inject({ method: 'POST', url: `/api/v1/agents/${link.agent_id}/token-rotation` })
+    const payload = { request_id: requested.json().request_id, next_token: token('revoked-next') }
+    const exchange = () => app.inject({ method: 'POST', url: '/v1/agent/token-rotation', headers: { authorization: 'Bearer ' + body.token }, payload })
+    expect((await exchange()).statusCode).toBe(200)
+    repos.updateAgent(app.db, link.agent_id, { status: 'revoked' })
+    expect((await exchange()).statusCode).toBe(401)
+  })
+})

apps/api/src/services/enrollment.ts

@@ -0,0 +1,100 @@
+import { createHash, createHmac, timingSafeEqual, randomUUID } from 'node:crypto'
+import type { Db, Sqlite } from '@evofw/db'
+import { repos } from '@evofw/db'
+import type { EnrollmentCredentialBody, TokenRotationExchangeBody } from '@evofw/shared'
+import { AppError } from '../plugins/error-handler.js'
+
+export const CREDENTIAL_TTL_MS = 24 * 60 * 60 * 1000
+export const CREDENTIAL_CLIENT_VERSIONS = new Set(['linux/10', 'rsc/10', 'install.sh/2'])
+const digest = (value: string) => createHash('sha256').update(value).digest('hex')
+const equal = (a: string, b: string) => timingSafeEqual(Buffer.from(digest(a)), Buffer.from(digest(b)))
+type Link = NonNullable<ReturnType<typeof repos.getInstallLink>>
+export function linkEnrollmentCredential(db: Db, link: Link, fallback: string): string {
+  const key = repos.getSetting(db, 'enroll_seed') || fallback
+  return createHmac('sha256', key).update(JSON.stringify(['evofw-enroll-v1', link.id, link.slug, link.agentId])).digest('base64url')
+}
+export function enrollWithCredential(db: Db, sqlite: Sqlite, fallback: string, body: EnrollmentCredentialBody, credential: string, ip: string) {
+  return sqlite.transaction(() => {
+    const link = repos.getInstallLink(db, body.install_link_id)
+    if (!link || !equal(credential, linkEnrollmentCredential(db, link, fallback))) throw new AppError('UNAUTHORIZED', 'Invalid enrollment credential', 401)
+    const lifecycle = sqlite.prepare('SELECT expires_at, consumed_at, consumed_token_hash FROM agent_install_links WHERE id=?').get(link.id) as { expires_at: string | null; consumed_at: string | null; consumed_token_hash: string | null }
+    const now = new Date().toISOString()
+    if (link.revokedAt || !lifecycle.expires_at || lifecycle.expires_at <= now) throw new AppError('GONE', 'Install link revoked or expired', 410)
+    const agent = link.agentId ? repos.getAgent(db, link.agentId) : undefined
+    if (!agent || agent.status === 'revoked') throw new AppError('GONE', 'Invited agent unavailable', 410)
+    const tokenHash = digest(body.token)
+    if (lifecycle.consumed_at) {
+      if (lifecycle.consumed_token_hash !== tokenHash || agent.tokenHash !== tokenHash) throw new AppError('CONFLICT', 'Install link already consumed', 409)
+      return { created: false, agent }
+    }
+    if (agent.status !== 'invited' || agent.platform !== body.platform) throw new AppError('CONFLICT', 'Invite does not match agent', 409)
+    const collision = repos.getAgentByTokenHash(db, tokenHash)
+    if (collision) throw new AppError('CONFLICT', 'Token already enrolled', 409)
+    const changed = sqlite.prepare('UPDATE agent_install_links SET consumed_at=?, consumed_token_hash=? WHERE id=? AND consumed_at IS NULL AND revoked_at IS NULL AND expires_at>?').run(now, tokenHash, link.id, now)
+    if (changed.changes !== 1) throw new AppError('CONFLICT', 'Install link already consumed', 409)
+    const updated = repos.updateAgent(db, agent.id, { tokenHash, tokenPrefix: body.token.slice(0, 12), status: 'pending', name: body.name, hostname: body.hostname ?? null, clientVersion: body.client_version ?? null, lastSeenAt: now, lastSeenIp: ip })!
+    return { created: true, agent: updated }
+  }).immediate()
+}
+interface Rotation { agent_id: string; request_id: string; expires_at: string; completed_at: string | null; previous_token_hash: string | null; next_token_hash: string | null }
+export function getCredentialStatus(sqlite: Sqlite, agentId: string) {
+  const row = sqlite.prepare('SELECT * FROM agent_token_rotations WHERE agent_id=?').get(agentId) as Rotation | undefined
+  const now = new Date().toISOString()
+  if (row?.completed_at && row.expires_at <= now) sqlite.prepare('UPDATE agent_token_rotations SET previous_token_hash=NULL WHERE agent_id=? AND expires_at<=?').run(agentId, now)
+  const receipt = sqlite.prepare("SELECT r.request_id FROM agent_credential_receipts r JOIN agents a ON a.id=r.agent_id WHERE r.agent_id=? AND r.token_hash=a.token_hash AND a.status='approved'").get(agentId) as { request_id: string } | undefined
+  return { client_id: agentId, rotation: row && !row.completed_at && row.expires_at > now ? { request_id: row.request_id, expires_at: row.expires_at } : null, completed_rotation: receipt ? { request_id: receipt.request_id } : null }
+}
+export function requestTokenRotation(db: Db, sqlite: Sqlite, agentId: string) {
+  return sqlite.transaction(() => {
+    const agent = repos.getAgent(db, agentId)
+    if (!agent) throw new AppError('NOT_FOUND', 'Agent not found', 404)
+    if (agent.status !== 'approved') throw new AppError('CONFLICT', 'Agent must be approved', 409)
+    if (!CREDENTIAL_CLIENT_VERSIONS.has(agent.clientVersion ?? '')) throw new AppError('UNSUPPORTED_CAPABILITY', 'Update agent before requesting token rotation', 409)
+    const active = sqlite.prepare('SELECT * FROM agent_token_rotations WHERE agent_id=?').get(agentId) as Rotation | undefined
+    const now = new Date().toISOString()
+    // Retain a completed exchange for its full retry window; do not strand a client after lost ACK.
+    if (active && active.expires_at > now) {
+      if (active.completed_at) throw new AppError('CONFLICT', 'Previous rotation recovery window still active', 409)
+      return { request_id: active.request_id, expires_at: active.expires_at }
+    }
+    const id = randomUUID(), expires = new Date(Date.now() + CREDENTIAL_TTL_MS).toISOString()
+    sqlite.prepare('INSERT INTO agent_token_rotations(agent_id,request_id,created_at,expires_at) VALUES(?,?,?,?) ON CONFLICT(agent_id) DO UPDATE SET request_id=excluded.request_id,created_at=excluded.created_at,expires_at=excluded.expires_at,completed_at=NULL,previous_token_hash=NULL,next_token_hash=NULL').run(agentId,id,now,expires)
+    return { request_id: id, expires_at: expires }
+  }).immediate()
+}
+export function exchangeToken(db: Db, sqlite: Sqlite, token: string, body: TokenRotationExchangeBody) {
+  return sqlite.transaction(() => {
+    const rotation = sqlite.prepare('SELECT * FROM agent_token_rotations WHERE request_id=?').get(body.request_id) as Rotation | undefined
+    const now = new Date().toISOString()
+    const agent = rotation ? repos.getAgent(db, rotation.agent_id) : undefined
+    if (!rotation || !agent || agent.status !== 'approved' || rotation.expires_at <= now) throw new AppError('UNAUTHORIZED', 'Invalid or expired token exchange', 401)
+    const current = digest(token), next = digest(body.next_token)
+    if (rotation.completed_at) {
+      if (agent.tokenHash !== rotation.next_token_hash || rotation.next_token_hash !== next || (current !== rotation.previous_token_hash && current !== rotation.next_token_hash)) throw new AppError('UNAUTHORIZED', 'Invalid token exchange retry', 401)
+    } else {
+      if (agent.tokenHash !== current) throw new AppError('UNAUTHORIZED', 'Invalid agent token', 401)
+      if (current === next || repos.getAgentByTokenHash(db, next)) throw new AppError('CONFLICT', 'New token must be unique', 409)
+      const changed = sqlite.prepare("UPDATE agents SET token_hash=?, token_prefix=? WHERE id=? AND token_hash=? AND status='approved'").run(next,body.next_token.slice(0,12),agent.id,current)
+      if (changed.changes !== 1) throw new AppError('CONFLICT', 'Agent credential changed', 409)
+      sqlite.prepare('UPDATE agent_token_rotations SET previous_token_hash=?,next_token_hash=?,completed_at=? WHERE request_id=?').run(current,next,now,body.request_id)
+      sqlite.prepare('INSERT INTO agent_credential_receipts(agent_id,request_id,token_hash,completed_at) VALUES(?,?,?,?) ON CONFLICT(agent_id) DO UPDATE SET request_id=excluded.request_id,token_hash=excluded.token_hash,completed_at=excluded.completed_at').run(agent.id,body.request_id,next,now)
+    }
+    return { ok: true as const, request_id: body.request_id, client_id: agent.id }
+  }).immediate()
+}
+
+/** Endpoint-specific identity only. Handler must still validate the exact exchange body. */
+export function authenticateTokenExchange(db: Db, sqlite: Sqlite, token: string): string | null {
+  const hash = digest(token)
+  const agent = repos.getAgentByTokenHash(db, hash)
+  if (agent?.status === 'approved') return agent.id
+  const row = sqlite.prepare("SELECT r.agent_id FROM agent_token_rotations r JOIN agents a ON a.id=r.agent_id WHERE r.previous_token_hash=? AND r.completed_at IS NOT NULL AND r.expires_at>? AND a.token_hash=r.next_token_hash AND a.status='approved'").get(hash, new Date().toISOString()) as { agent_id: string } | undefined
+  return row?.agent_id ?? null
+}
+
+export function getOperatorCredentialStatus(db: Db, sqlite: Sqlite, agentId: string) {
+  const agent = repos.getAgent(db, agentId)
+  if (!agent) throw new AppError('NOT_FOUND', 'Agent not found', 404)
+  const status = getCredentialStatus(sqlite, agentId)
+  return { agent_id: agentId, supported: CREDENTIAL_CLIENT_VERSIONS.has(agent.clientVersion ?? ''), client_version: agent.clientVersion, agent_status: agent.status, rotation: status.rotation, completed_rotation: status.completed_rotation }
+}

apps/api/src/services/evobgp.ts

@@ -0,0 +1,41 @@
+import { safeFetchJson, SOURCE_LIMITS } from './safe-fetch.js'
+
+/** Fully collect bounded cursor pages. Never accept an incomplete snapshot. */
+export async function fetchEvobgpPages(base: string, token: string, path: string, limit: number): Promise<Record<string, unknown>[]> {
+  const pages: Record<string, unknown>[] = []
+  const seen = new Set<string>()
+  let cursor: string | undefined
+  let elements = 0
+  let bytes = 0
+  const deadline = Date.now() + SOURCE_LIMITS.timeoutMs
+  for (let page = 0; page < SOURCE_LIMITS.pages; page++) {
+    if (Date.now() >= deadline) throw new Error('EvoBGP snapshot deadline exceeded')
+    if (bytes >= SOURCE_LIMITS.totalBytes) throw new Error('EvoBGP snapshot byte limit exceeded')
+    const url = new URL(`${base.replace(/\/$/, '')}${path}`)
+    url.searchParams.set('limit', String(limit))
+    if (cursor) url.searchParams.set('cursor', cursor)
+    const response = await safeFetchJson(url, {
+      headers: { Authorization: `Bearer ${token}`, Accept: 'application/json' },
+      timeoutMs: deadline - Date.now(),
+      maxBytes: Math.min(SOURCE_LIMITS.bytes, SOURCE_LIMITS.totalBytes - bytes),
+    })
+    const data = response.data
+    bytes += response.bytes
+    if (bytes > SOURCE_LIMITS.totalBytes) throw new Error('EvoBGP snapshot byte limit exceeded')
+    if (!data || typeof data !== 'object' || Array.isArray(data)) throw new Error('Invalid EvoBGP page')
+    const body = data as Record<string, unknown>
+    const items = body.prefixes ?? body.items
+    if (!Array.isArray(items)) throw new Error('Invalid EvoBGP page items')
+    elements += items.length
+    if (elements > SOURCE_LIMITS.elements) throw new Error('EvoBGP snapshot element limit exceeded')
+    if (body.has_more !== undefined && typeof body.has_more !== 'boolean') throw new Error('Invalid EvoBGP has_more')
+    if (body.next_cursor !== undefined && body.next_cursor !== null && (typeof body.next_cursor !== 'string' || body.next_cursor.length > 2048)) throw new Error('Invalid EvoBGP next_cursor')
+    pages.push(body)
+    if (body.has_more !== true && !body.next_cursor) return pages
+    const next = body.next_cursor
+    if (typeof next !== 'string' || !next || seen.has(next)) throw new Error('Incomplete or repeated EvoBGP cursor')
+    seen.add(next)
+    cursor = next
+  }
+  throw new Error('EvoBGP page limit exceeded')
+}

apps/api/src/services/install-links.test.ts

@@ -1,3 +1,5 @@
+import { repos } from '@evofw/db'
+import { linkEnrollmentCredential } from './enrollment.js'
 import { describe, it, expect, afterAll } from 'vitest'
 import { buildApp } from '../app.js'
 import type { AppConfig } from '../config.js'
@@ -95,7 +97,7 @@ describe('install-links', () => {
       url: '/v1/agent/enroll',
       headers: {
         'content-type': 'application/json',
-        'x-evofw-seed': 'test-seed',
+        'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
       },
       payload: {
         name: 'web-02',
@@ -217,7 +219,7 @@ describe('install-links', () => {
       url: '/v1/agent/enroll',
       headers: {
         'content-type': 'application/json',
-        'x-evofw-seed': 'test-seed',
+        'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
       },
       payload: {
         name: 'empty-policy',
@@ -280,7 +282,7 @@ describe('install-links', () => {
       url: '/v1/agent/enroll',
       headers: {
         'content-type': 'application/json',
-        'x-evofw-seed': 'test-seed',
+        'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
       },
       payload: {
         name: 'mt-policy',
@@ -358,7 +360,7 @@ describe('install-links', () => {
       url: '/v1/agent/enroll',
       headers: {
         'content-type': 'application/json',
-        'x-evofw-seed': 'test-seed',
+        'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
       },
       payload: {
         name: 'script-sha-policy',

apps/api/src/services/install-links.ts

@@ -1,3 +1,4 @@
+import { linkEnrollmentCredential } from './enrollment.js'
 import { readFileSync } from 'node:fs'
 import { join } from 'node:path'
 import type { Db } from '@evofw/db'
@@ -90,6 +91,8 @@ export function mapInstallLink(
     agent_id: row.agentId ?? null,
     created_at: row.createdAt,
     revoked_at: row.revokedAt,
+    expires_at: row.expiresAt,
+    consumed_at: row.consumedAt,
     last_used_at: row.lastUsedAt,
     use_count: row.useCount,
     urls: { by_id: urls.by_id, by_slug: urls.by_slug },
@@ -124,7 +127,7 @@ function escapeRosString(s: string): string {
  */
 export function renderInstallScript(opts: {
   cpUrl: string
-  seed: string
+  credential: string
   clientName: string
   platform: string
   installLinkId: string
@@ -136,7 +139,7 @@ export function renderInstallScript(opts: {
     '# EvoFirewall short install link — env pre-set',
     'set -euo pipefail',
     `export EVOFW_CP_URL='${escape(cp)}'`,
-    `export EVOFW_SEED='${escape(opts.seed)}'`,
+    `export EVOFW_ENROLLMENT_CREDENTIAL='${escape(opts.credential)}'`,
     `export EVOFW_CLIENT_NAME='${escape(opts.clientName)}'`,
     `export EVOFW_PLATFORM='${escape(opts.platform)}'`,
     `export EVOFW_INSTALL_LINK_ID='${escape(opts.installLinkId)}'`,
@@ -153,7 +156,7 @@ export function renderInstallScript(opts: {
  */
 export function renderMikrotikInstallScript(opts: {
   cpUrl: string
-  seed: string
+  credential: string
   clientName: string
   installLinkId: string
 }): string {
@@ -161,7 +164,7 @@ export function renderMikrotikInstallScript(opts: {
   const header = [
     '# EvoFirewall short install link — globals pre-set (RouterOS 7.21+)',
     `:global EvofwCpUrl "${escapeRosString(cp)}"`,
-    `:global EvofwSeed "${escapeRosString(opts.seed)}"`,
+    `:global EvofwEnrollmentCredential "${escapeRosString(opts.credential)}"`,
     `:global EvofwName "${escapeRosString(opts.clientName)}"`,
     `:global EvofwInstallLinkId "${escapeRosString(opts.installLinkId)}"`,
     '',
@@ -194,13 +197,16 @@ export function resolveAndRenderInstall(
   if (link.revokedAt) {
     throw new AppError('GONE', 'Install link revoked', 410)
   }
-  const seed = repos.getSetting(db, 'enroll_seed') || enrollSeedFallback
+  if (!link.expiresAt || link.expiresAt <= new Date().toISOString()) {
+    throw new AppError('GONE', 'Install link expired', 410)
+  }
+  const credential = linkEnrollmentCredential(db, link, enrollSeedFallback)
   repos.touchInstallLink(db, link.id)
   if (link.platform === 'mikrotik') {
     return {
       body: renderMikrotikInstallScript({
         cpUrl: publicBaseUrl,
-        seed,
+        credential,
         clientName: link.clientName,
         installLinkId: link.id,
       }),
@@ -210,7 +216,7 @@ export function resolveAndRenderInstall(
   return {
     body: renderInstallScript({
       cpUrl: publicBaseUrl,
-      seed,
+      credential,
       clientName: link.clientName,
       platform: link.platform,
       installLinkId: link.id,

apps/api/src/services/ip-block-stats.test.ts

@@ -1,6 +1,8 @@
 import { describe, it, expect, afterAll } from 'vitest'
 import { buildApp } from '../app.js'
 import type { AppConfig } from '../config.js'
+import { repos } from '@evofw/db'
+import { linkEnrollmentCredential } from './enrollment.js'
 
 const testConfig: AppConfig = {
   databaseUrl: 'sqlite::memory:',
@@ -38,7 +40,7 @@ async function enrollApprovedLinux(
     url: '/v1/agent/enroll',
     headers: {
       'content-type': 'application/json',
-      'x-evofw-seed': 'test-seed',
+      'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
     },
     payload: {
       name,
@@ -274,7 +276,7 @@ describe('apply-report ip_hits / blocked-ips', () => {
       url: '/v1/agent/enroll',
       headers: {
         'content-type': 'application/json',
-        'x-evofw-seed': 'test-seed',
+        'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
       },
       payload: {
         name: 'mt-hits',
@@ -348,7 +350,6 @@ describe('apply-report ip_hits / blocked-ips', () => {
     expect(body2.items[0]!.last_seen_at >= firstSeen).toBe(true)
 
     // Re-entry after stale gap (> PRESENCE_REHIT_STALE_MS): simulate via future `now`
-    const { repos } = await import('@evofw/db')
     repos.upsertIpBlockStats(
       app.db,
       link.agent_id,

apps/api/src/services/labels.test.ts

@@ -0,0 +1,117 @@
+import { afterEach, describe, expect, it } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { previewLabelAssignmentSchema, applyLabelAssignmentSchema, permissionForRequest } from '@evofw/shared'
+import { createAgentLabel, patchAgentLabel, putAgentLabels, previewLabelAssignment, applyLabelAssignment } from './labels.js'
+import { createPolicySnapshot, startPolicyRollout } from './policy/rollouts.js'
+import { buildApp } from '../app.js'
+import { loadConfig } from '../config.js'
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+afterEach(() => { for (const mem of opened.splice(0)) mem.sqlite.close() })
+function memory(count = 3) {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  for (let index = 0; index < count; index++) repos.insertAgent(mem.db, { id: `a${index}`, name: `Agent ${index}`, tokenHash: `t${index}`, tokenPrefix: `t${index}`, status: 'approved', telemetryVersion: 2, capabilitiesJson: '["policy-ipv4"]' })
+  for (const id of ['old', 'new', 'third']) repos.insertPolicySet(mem.db, { id, name: id })
+  return mem
+}
+describe('labels and materialized group assignment', () => {
+  it('treats labels as metadata, uses normalized unique names and atomically validates replacement', () => {
+    const { db } = memory()
+    const label = createAgentLabel(db, { name: 'Production', color: 'success' })
+    expect(() => createAgentLabel(db, { name: 'PRODUCTION' })).toThrow('already exists')
+    putAgentLabels(db, 'a0', [label.id])
+    patchAgentLabel(db, label.id, { name: 'Production fleet' })
+    expect(repos.getAgent(db, 'a0')!.policyGeneration).toBe(1)
+    expect(() => putAgentLabels(db, 'a0', ['missing'])).toThrow('Label not found')
+    expect(repos.agentLabelIds(db, 'a0')).toEqual([label.id])
+    repos.deleteAgentLabel(db, label.id)
+    expect(repos.agentLabelIds(db, 'a0')).toEqual([])
+    expect(repos.getAgent(db, 'a0')!.policyGeneration).toBe(1)
+  })
+
+  it('preserves existing order in add, performs exact replace and skips identical assignments', () => {
+    const { db } = memory()
+    repos.setAgentPolicySets(db, 'a0', ['old', 'third'])
+    const request = { selector: { agent_ids: ['a0', 'a1'] }, mode: 'add' as const, set_ids: ['third', 'new'] }
+    const preview = previewLabelAssignment(db, request)
+    expect(preview.items[0]!.set_ids).toEqual(['old', 'third', 'new'])
+    const result = applyLabelAssignment(db, { ...request, expected: preview.items.map((row) => ({ agent_id: row.agent_id, generation: row.generation })) })
+    expect(result.affected_agent_ids).toEqual(['a0', 'a1'])
+    const again = previewLabelAssignment(db, request)
+    const unchanged = applyLabelAssignment(db, { ...request, expected: again.items.map((row) => ({ agent_id: row.agent_id, generation: row.generation })) })
+    expect(unchanged.affected_agent_ids).toEqual([])
+    expect(unchanged.items.map((row) => row.generation)).toEqual(again.items.map((row) => row.generation))
+    const replace = { selector: { agent_ids: ['a0'] }, mode: 'replace' as const, set_ids: ['new'] }
+    const replacement = previewLabelAssignment(db, replace)
+    applyLabelAssignment(db, { ...replace, expected: replacement.items.map((row) => ({ agent_id: row.agent_id, generation: row.generation })) })
+    expect(repos.listSetsForAgent(db, 'a0').map((row) => row.setId)).toEqual(['new'])
+  })
+
+  it('requires exact materialized membership and generations; later labels never autoassign', () => {
+    const { db } = memory()
+    const label = createAgentLabel(db, { name: 'Canaries' })
+    putAgentLabels(db, 'a0', [label.id])
+    const request = { selector: { label_ids: [label.id] }, mode: 'replace' as const, set_ids: ['new'] }
+    const preview = previewLabelAssignment(db, request)
+    const expected = preview.items.map((row) => ({ agent_id: row.agent_id, generation: row.generation }))
+    putAgentLabels(db, 'a1', [label.id])
+    expect(() => applyLabelAssignment(db, { ...request, expected })).toThrow('membership')
+    expect(repos.listSetsForAgent(db, 'a0')).toHaveLength(0)
+    putAgentLabels(db, 'a1', [])
+    applyLabelAssignment(db, { ...request, expected })
+    putAgentLabels(db, 'a1', [label.id])
+    expect(repos.listSetsForAgent(db, 'a1')).toHaveLength(0)
+    expect(() => applyLabelAssignment(db, { ...request, expected })).toThrow('generation')
+  })
+
+  it('rejects invalid sets and pinned targets before writing any agent', () => {
+    const { db } = memory()
+    const request = { selector: { agent_ids: ['a0', 'a1'] }, mode: 'replace' as const, set_ids: ['new'] }
+    const preview = previewLabelAssignment(db, request)
+    const expected = preview.items.map((row) => ({ agent_id: row.agent_id, generation: row.generation }))
+    expect(() => applyLabelAssignment(db, { ...request, set_ids: ['new', 'missing'], expected })).toThrow('Policy set not found')
+    expect(repos.listSetsForAgent(db, 'a0')).toHaveLength(0)
+    const snapshot = createPolicySnapshot(db, 'a2', 'Empty')
+    startPolicyRollout(db, { snapshot_id: snapshot.id, targets: [{ agent_id: 'a1', expected_generation: 1 }], canary_ids: ['a1'] })
+    expect(() => applyLabelAssignment(db, { ...request, expected })).toThrow('Release')
+    expect(repos.listSetsForAgent(db, 'a0')).toHaveLength(0)
+    expect(repos.getAgent(db, 'a0')!.policyGeneration).toBe(1)
+  })
+
+  it('supports exactly 100 selected agents and rejects a 101-agent label group without truncation', () => {
+    const { db } = memory(101)
+    const label = createAgentLabel(db, { name: 'Large group' })
+    for (let index = 0; index < 100; index++) putAgentLabels(db, `a${index}`, [label.id])
+    const request = { selector: { label_ids: [label.id] }, mode: 'add' as const, set_ids: ['new'] }
+    const preview = previewLabelAssignment(db, request)
+    expect(preview.total).toBe(100)
+    const applied = applyLabelAssignment(db, { ...request, expected: preview.items.map((row) => ({ agent_id: row.agent_id, generation: row.generation })) })
+    expect(applied.affected_agent_ids).toHaveLength(100)
+    expect(repos.listSetsForAgent(db, 'a100')).toHaveLength(0)
+    putAgentLabels(db, 'a100', [label.id])
+    expect(() => previewLabelAssignment(db, request)).toThrow('100 agents')
+    expect(repos.listSetsForAgent(db, 'a100')).toHaveLength(0)
+    expect(previewLabelAssignmentSchema.safeParse({ ...request, selector: { agent_ids: ['a0'], label_ids: [label.id] } }).success).toBe(false)
+    expect(applyLabelAssignmentSchema.safeParse({ ...request, expected: [{ agent_id: 'a0', generation: 1 }, { agent_id: 'a0', generation: 1 }] }).success).toBe(false)
+  })
+
+  it('provides HTTP label CRUD, AND fleet filtering and permission separation', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } })
+    try {
+      await app.ready()
+      for (const id of ['a', 'b']) repos.insertAgent(app.db, { id, name: id, tokenPrefix: id, tokenHash: id })
+      const one = await app.inject({ method: 'POST', url: '/api/v1/agent-labels', payload: { name: 'One' } })
+      const two = await app.inject({ method: 'POST', url: '/api/v1/agent-labels', payload: { name: 'Two' } })
+      expect(one.statusCode).toBe(201)
+      expect(one.json().name_key).toBeUndefined()
+      await app.inject({ method: 'PUT', url: '/api/v1/agents/a/labels', payload: { label_ids: [one.json().id, two.json().id] } })
+      await app.inject({ method: 'PUT', url: '/api/v1/agents/b/labels', payload: { label_ids: [one.json().id] } })
+      const filtered = await app.inject({ url: `/api/v1/agents?label_ids=${one.json().id},${two.json().id}` })
+      expect(filtered.json().items.map((row: { id: string }) => row.id)).toEqual(['a'])
+      expect((await app.inject({ url: '/api/v1/agents?label_ids=' })).json().items).toHaveLength(2)
+      expect(permissionForRequest('POST', '/api/v1/agent-labels/preview-assignment')).toBe('fw:policies:read')
+      expect(permissionForRequest('POST', '/api/v1/agent-labels/assign-policy-sets')).toBe('fw:policies:write')
+      expect(permissionForRequest('POST', '/api/v1/agent-labels')).toBe('fw:agents:write')
+    } finally { await app.close() }
+  })
+})

apps/api/src/services/labels.ts

@@ -0,0 +1,66 @@
+import { randomUUID } from 'node:crypto'
+import { repos, type Db } from '@evofw/db'
+import type { PreviewLabelAssignment, ApplyLabelAssignment, LabelAssignmentPreview } from '@evofw/shared'
+import { AppError } from '../plugins/error-handler.js'
+const fail = (code: string, message: string, status = 409): never => { throw new AppError(code, message, status) }
+const requireAgent = (db: Db, id: string) => repos.getAgent(db, id) ?? fail('NOT_FOUND', 'Agent not found', 404)
+const requireLabel = (db: Db, id: string) => repos.getAgentLabel(db, id) ?? fail('NOT_FOUND', 'Label not found', 404)
+const nameKey = (name: string) => name.normalize('NFKC').toLocaleLowerCase('en-US')
+export function createAgentLabel(db: Db, body: { name: string; color?: string | null }, now = new Date().toISOString()) {
+  return db.transaction((tx) => {
+    const key = nameKey(body.name)
+    if (repos.getAgentLabelByName(tx, key)) fail('LABEL_NAME_CONFLICT', 'A label with this name already exists')
+    if (repos.countAgentLabels(tx) >= 1000) fail('LABEL_CAPACITY', 'At most 1000 labels may be stored', 422)
+    const row = { id: randomUUID(), name: body.name, name_key: key, color: body.color ?? null, created_at: now }
+    repos.insertAgentLabel(tx, row)
+    return row
+  })
+}
+export function patchAgentLabel(db: Db, id: string, patch: { name?: string; color?: string | null }) {
+  return db.transaction((tx) => {
+    const previous = requireLabel(tx, id)
+    const name = patch.name ?? previous.name
+    const key = nameKey(name)
+    const conflict = repos.getAgentLabelByName(tx, key)
+    if (conflict && conflict.id !== id) fail('LABEL_NAME_CONFLICT', 'A label with this name already exists')
+    const row = { ...previous, name, name_key: key, color: patch.color === undefined ? previous.color : patch.color }
+    repos.updateAgentLabel(tx, row)
+    return row
+  })
+}
+export function putAgentLabels(db: Db, id: string, labelIds: string[]) {
+  return db.transaction((tx) => {
+    requireAgent(tx, id)
+    for (const labelId of labelIds) requireLabel(tx, labelId)
+    repos.replaceAgentLabels(tx, id, labelIds)
+    return { agent_id: id, label_ids: repos.agentLabelIds(tx, id) }
+  })
+}
+export function previewLabelAssignment(db: Db, body: PreviewLabelAssignment): LabelAssignmentPreview {
+  // Resolve the whole selection once. Never silently cap a larger label group.
+  for (const labelId of body.selector.label_ids ?? []) requireLabel(db, labelId)
+  const ids = body.selector.agent_ids ?? repos.agentIdsWithLabels(db, body.selector.label_ids!, 101)
+  if (ids.length > 100) fail('GROUP_TOO_LARGE', 'An assignment may select at most 100 agents', 422)
+  for (const setId of body.set_ids) if (!repos.getPolicySet(db, setId)) fail('NOT_FOUND', 'Policy set not found', 404)
+  const items = ids.map((id) => {
+    const agent = requireAgent(db, id)
+    repos.assertAgentPolicyMutable(db, id)
+    const previous = repos.listSetsForAgent(db, id).map((set) => set.setId)
+    const final = body.mode === 'replace' ? [...body.set_ids] : [...previous, ...body.set_ids.filter((setId) => !previous.includes(setId))]
+    if (final.length > 100) fail('ASSIGNMENT_TOO_LARGE', 'An agent may receive at most 100 policy sets through a group assignment', 422)
+    return { agent_id: agent.id, generation: agent.policyGeneration, previous_set_ids: previous, set_ids: final, changed: JSON.stringify(previous) !== JSON.stringify(final) }
+  })
+  return { mode: body.mode, items, total: items.length }
+}
+export function applyLabelAssignment(db: Db, body: ApplyLabelAssignment) {
+  return db.transaction((tx) => {
+    const preview = previewLabelAssignment(tx, body)
+    if (preview.total === 0) fail('EMPTY_SELECTION', 'No agents match this selection')
+    const expected = new Map(body.expected.map((row) => [row.agent_id, row.generation]))
+    if (expected.size !== preview.total || preview.items.some((item) => expected.get(item.agent_id) !== item.generation)) fail('REVISION_CONFLICT', 'Group membership or an agent generation changed; refresh preview')
+    // Every target/set/pin/precondition has been validated before the first write.
+    for (const item of preview.items) if (item.changed) repos.setAgentPolicySets(tx, item.agent_id, item.set_ids)
+    return { affected_agent_ids: preview.items.filter((item) => item.changed).map((item) => item.agent_id),
+      items: preview.items.map((item) => ({ agent_id: item.agent_id, generation: requireAgent(tx, item.agent_id).policyGeneration, before_count: item.previous_set_ids.length, after_count: item.set_ids.length, changed: item.changed })) }
+  })
+}

apps/api/src/services/list-import-summary.test.ts

@@ -0,0 +1,108 @@
+import { describe, expect, it } from 'vitest'
+import { repos } from '@evofw/db'
+import { adminCredentialStatusSchema, listImportSummarySchema } from '@evofw/shared'
+import { buildApp } from '../app.js'
+import { loadConfig } from '../config.js'
+
+describe('bounded opt-in list import', () => {
+  it('returns a bounded 100000-entry summary, keeps no-op generation stable and rejects mixed invalid imports atomically', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } })
+    try {
+      await app.ready()
+      repos.insertIpList(app.db, { id: 'large', name: 'Large', type: 'static' })
+      repos.insertAgent(app.db, { id: 'a', name: 'A', tokenHash: 'a', tokenPrefix: 'a', status: 'approved' })
+      repos.insertPolicySet(app.db, { id: 'set', name: 'Set' })
+      repos.insertPolicyRule(app.db, { id: 'r', setId: 'set', action: 'deny', listId: 'large', priority: 10 })
+      repos.setAgentPolicySets(app.db, 'a', ['set'])
+      const values = Array.from({ length: 100_000 }, (_, index) => `10.${Math.floor(index / 65536)}.${Math.floor(index / 256) % 256}.${index % 256}`)
+      const started = performance.now()
+      const imported = await app.inject({ method: 'POST', url: '/api/v1/lists/large/entries?return=summary', payload: { values } })
+      expect(imported.statusCode).toBe(200)
+      const summary = listImportSummarySchema.parse(imported.json())
+      expect(summary.count).toBe(100_000)
+      expect(summary.content_hash).toMatch(/^sha256:[a-f0-9]{64}$/)
+      expect(summary.changed).toBe(true)
+      expect(Buffer.byteLength(imported.body)).toBeLessThan(256)
+      console.info(`100000-entry import summary: ${Buffer.byteLength(imported.body)} bytes, ${(performance.now() - started).toFixed(1)}ms (Windows/in-memory)`)
+      const page = await app.inject({ url: '/api/v1/lists/large?view=page&limit=200&offset=99800' })
+      expect(page.statusCode).toBe(200)
+      expect(page.json().items).toHaveLength(200)
+      expect(page.json().total).toBe(100_000)
+      expect(page.json().entry_count).toBe(100_000)
+      expect(page.json().config_json).toBe('{}')
+      expect(Buffer.byteLength(page.body)).toBeLessThan(100_000)
+      const filters = JSON.stringify([{ field: 'value', operator: 'is', values: [values[99_999]!] }, { field: 'kind', operator: 'is', values: ['ip'] }])
+      const filtered = await app.inject({ url: `/api/v1/lists/large?view=page&filters=${encodeURIComponent(filters)}` })
+      expect(filtered.json().total).toBe(1)
+      expect(filtered.json().items[0]!.value).toBe(values[99_999])
+      const catalog = await app.inject({ url: '/api/v1/lists?view=summary' })
+      expect(Buffer.byteLength(catalog.body)).toBeLessThan(1000)
+      expect(catalog.json().items[0].config_json).toBe('{}')
+      console.info(`100000-entry detail page: ${Buffer.byteLength(page.body)} bytes for 200 items; filtered late match ${Buffer.byteLength(filtered.body)} bytes`)
+      const revision = repos.getAgent(app.db, 'a')!.policyGeneration
+      const duplicate = await app.inject({ method: 'POST', url: '/api/v1/lists/large/entries?return=summary', payload: { values } })
+      expect(duplicate.statusCode).toBe(200)
+      expect(duplicate.json()).toEqual({ ...summary, changed: false })
+      expect(repos.getAgent(app.db, 'a')!.policyGeneration).toBe(revision)
+      const invalid = await app.inject({ method: 'POST', url: '/api/v1/lists/large/entries?return=summary', payload: { values: ['192.0.2.100', '192.0.2.999'] } })
+      expect(invalid.statusCode).toBe(400)
+      expect(repos.getIpListEntrySummary(app.db, 'large')).toEqual({ count: 100_000, content_hash: summary.content_hash })
+      expect(repos.getAgent(app.db, 'a')!.policyGeneration).toBe(revision)
+      const legacyDuplicate = await app.inject({ method: 'POST', url: '/api/v1/lists/large/entries', payload: { values: [values[0]!] } })
+      expect(legacyDuplicate.statusCode).toBe(400)
+      const deleted = await app.inject({ method: 'DELETE', url: '/api/v1/lists/large/entries?return=summary', payload: { value: values[99_999] } })
+      expect(deleted.statusCode).toBe(200)
+      expect(deleted.json()).toMatchObject({ count: 99_999, changed: true })
+      expect(Buffer.byteLength(deleted.body)).toBeLessThan(256)
+    } finally { await app.close() }
+  }, 30_000)
+
+  it('preserves the default legacy detail response and rejects unsupported opt-in values', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } })
+    try {
+      await app.ready()
+      repos.insertIpList(app.db, { id: 'small', name: 'Small', type: 'static' })
+      const legacy = await app.inject({ method: 'POST', url: '/api/v1/lists/small/entries', payload: { values: ['192.0.2.1'] } })
+      expect(legacy.statusCode).toBe(200)
+      expect(legacy.json().entries).toEqual(['192.0.2.1/32'])
+      expect(legacy.json().items).toHaveLength(1)
+      const bad = await app.inject({ method: 'POST', url: '/api/v1/lists/small/entries?return=everything', payload: { values: ['192.0.2.2'] } })
+      expect(bad.statusCode).toBe(400)
+      expect(repos.getIpListEntrySummary(app.db, 'small')!.count).toBe(1)
+      for (const query of ['view=page&limit=0', 'view=page&limit=201', 'view=page&offset=-1', 'view=page&limit=oops', 'limit=100', 'view=page&filters=bad', 'view=page&filters=%5B%7B%22field%22%3A%22secret%22%7D%5D']) {
+        expect((await app.inject({ url: `/api/v1/lists/small?${query}` })).statusCode).toBe(400)
+      }
+      expect(adminCredentialStatusSchema.safeParse({ agent_id: 'a', supported: false, client_version: null, agent_status: 'pending', rotation: null, completed_rotation: null }).success).toBe(true)
+    } finally { await app.close() }
+  })
+
+  it('pages external CIDRs and bounds nested resolved previews with whole-list Unicode name filtering', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } })
+    try {
+      await app.ready()
+      repos.insertIpList(app.db, { id: 'child', name: 'ПРОДУКТИВНЫЙ Источник', type: 'json_url' })
+      const cidrs = Array.from({ length: 500 }, (_, index) => `198.51.${Math.floor(index / 256)}.${index % 256}/32`)
+      repos.replaceIpListEntries(app.db, 'child', cidrs)
+      const external = await app.inject({ url: '/api/v1/lists/child?view=page&offset=490&limit=10&q=198.51.1.' })
+      expect(external.json().total).toBe(244)
+      expect(external.json().items).toHaveLength(0)
+      const last = await app.inject({ url: '/api/v1/lists/child?view=page&q=198.51.1.243%2F32' })
+      expect(last.json().total).toBe(1)
+      repos.insertIpList(app.db, { id: 'parent', name: 'Parent', type: 'static' })
+      await app.inject({ method: 'POST', url: '/api/v1/lists/parent/entries?return=summary', payload: { items: [{ kind: 'list', value: 'child' }] } })
+      const filters = encodeURIComponent(JSON.stringify([{ field: 'value', operator: 'contains', values: ['продуктивный'] }]))
+      const parent = await app.inject({ url: `/api/v1/lists/parent?view=page&filters=${filters}` })
+      expect(parent.statusCode).toBe(200)
+      expect(parent.json().total).toBe(1)
+      expect(parent.json().items[0]).toMatchObject({ resolved_count: 500, resolved_truncated: true })
+      expect(parent.json().items[0].resolved_cidrs).toHaveLength(3)
+      expect(Buffer.byteLength(parent.body)).toBeLessThan(3000)
+      repos.insertIpList(app.db, { id: 'legacy', name: 'Legacy', type: 'domains', configJson: JSON.stringify({ domains: ['192.0.2.1', 'Example.test'] }) })
+      const legacy = await app.inject({ url: '/api/v1/lists/legacy?view=page' })
+      expect(legacy.statusCode).toBe(200)
+      expect(legacy.json().items.map((item: { kind: string }) => item.kind)).toEqual(['hostname', 'hostname'])
+      repos.insertIpList(app.db, { id: 'malformed', name: 'Malformed', type: 'static', configJson: '{' })
+      expect((await app.inject({ url: '/api/v1/lists/malformed?view=page' })).json().items).toEqual([])
+    } finally { await app.close() }
+  })
+})

apps/api/src/services/lists/entries.ts

@@ -7,12 +7,18 @@ import {
   parseListEntry,
   readManualItems,
   splitListPlaintext,
+  canonicalHostname,
+  canonicalIp,
+  canonicalCidr,
   type ListConfigItem,
   type ListEntryInput,
+  type ListImportSummary,
+  type ListDetailPageQuery,
 } from '@evofw/shared'
 import { resolveHostnameToCidrs } from '../policy/resolve-hostname.js'
 import { uniqCidrs } from '../uniq.js'
 import { maskListConfig } from '../secret-cipher.js'
+import { createHash } from 'node:crypto'
 
 export function getListConfig(list: {
   configJson: string
@@ -60,109 +66,103 @@ export function findParentListIds(db: Db, childId: string): string[] {
   return parents
 }
 
-async function expandItem(
+type ListRevision = { configJson: string; type: string; contentHash: string | null; updatedAt: string }
-  db: Db,
+type ListPublication = { cidrs?: string[]; configJson?: string; type?: string; expected?: ListRevision }
-  item: ListConfigItem,
+
-  visiting: Set<string>,
+/** Resolve the whole dependency closure before publishing any config or CIDRs. */
-): Promise<string[]> {
+export async function publishListCascade(db: Db, listId: string, root: ListPublication = {}): Promise<string[]> {
-  if (item.kind === 'hostname') {
+  const snapshot = repos.listIpLists(db)
-    if (item.resolved_cidrs?.length) return item.resolved_cidrs
+  const lists = new Map(snapshot.map((list) => [list.id, list]))
-    return resolveHostnameToCidrs(item.value)
+  const initial = lists.get(listId)
+  if (!initial) throw new Error('List not found')
+  if (root.expected && (['configJson', 'type', 'contentHash', 'updatedAt'] as const).some(key => initial[key] !== root.expected![key])) {
+    throw new Error('List changed during source refresh; retry refresh')
   }
-  if (item.kind === 'list') {
+  lists.set(listId, { ...initial, ...(root.configJson === undefined ? {} : { configJson: root.configJson }), ...(root.type === undefined ? {} : { type: root.type }) })
-    const childId = item.value
+  const closure = new Set([listId])
-    if (visiting.has(childId)) {
+  let expanded = true
-      throw new Error(`Циклическая ссылка списков: ${childId}`)
+  while (expanded) {
+    expanded = false
+    for (const list of lists.values()) {
+      if (closure.has(list.id) || !isManualListType(list.type)) continue
+      if (listNestedChildIds(list.configJson).some((id) => closure.has(id))) {
+        closure.add(list.id)
+        expanded = true
       }
-    const child = repos.getIpList(db, childId)
-    if (!child) {
-      throw new Error(`Вложенный список не найден: ${childId}`)
     }
-    // Prefer materialized CIDRs; for manual children rebuild if empty.
-    let entries = repos.listIpListEntries(db, childId).map((e) => e.cidr)
-    if (entries.length === 0 && isManualListType(child.type)) {
-      entries = await rebuildManualListEntries(db, childId, visiting)
   }
-    return entries
+  const staged = new Map<string, { cidrs: string[]; configJson: string; type: string }>()
+  const visiting = new Set<string>()
+  const stage = async (id: string): Promise<string[]> => {
+    if (visiting.has(id)) throw new Error(`Циклическая ссылка списков: ${id}`)
+    const cached = staged.get(id)
+    if (cached) return cached.cidrs
+    const list = lists.get(id)
+    if (!list) throw new Error(`Вложенный список не найден: ${id}`)
+    visiting.add(id)
+    try {
+      let cidrs: string[]
+      let configJson = list.configJson
+      if (id === listId && root.cidrs !== undefined) cidrs = uniqCidrs(root.cidrs)
+      else if (isManualListType(list.type)) {
+        const config = getListConfig(list)
+        const items: ListConfigItem[] = []
+        const all: string[] = []
+        for (const item of readManualItems(list.configJson)) {
+          let resolved: string[]
+          if (item.kind === 'hostname') resolved = await resolveHostnameToCidrs(item.value)
+          else if (item.kind === 'list') {
+            const child = lists.get(item.value)
+            if (!child) throw new Error(`Вложенный список не найден: ${item.value}`)
+            const materialized = repos.listIpListEntries(db, child.id).map((entry) => entry.cidr)
+            resolved = closure.has(child.id) || (isManualListType(child.type) && materialized.length === 0)
+              ? await stage(child.id) : materialized
+          } else resolved = normalizeItemToCidrs(item)
+          items.push({ ...item, resolved_cidrs: resolved })
+          for (const cidr of resolved) all.push(cidr)
         }
-  return normalizeItemToCidrs(item)
+        config.items = items
+        delete config.domains
+        configJson = JSON.stringify(config)
+        cidrs = uniqCidrs(all)
+      } else cidrs = uniqCidrs(repos.listIpListEntries(db, id).map((entry) => entry.cidr))
+      staged.set(id, { cidrs, configJson, type: list.type })
+      return cidrs
+    } finally { visiting.delete(id) }
+  }
+  for (const id of closure) await stage(id)
+  const changed: string[] = []
+  db.transaction((tx) => {
+    const current = repos.listIpLists(tx)
+    if (current.length !== snapshot.length || snapshot.some((before) => {
+      const now = current.find((list) => list.id === before.id)
+      return !now || now.configJson !== before.configJson || now.contentHash !== before.contentHash || now.updatedAt !== before.updatedAt
+    })) throw new Error('Lists changed while resolving; retry refresh')
+    for (const [id, value] of staged) {
+      if (repos.replaceIpListEntries(tx, id, value.cidrs)) changed.push(id)
+      repos.updateIpList(tx, id, {
+        type: value.type,
+        configJson: value.configJson,
+        contentHash: `sha256:${createHash('sha256').update(value.cidrs.join('\n')).digest('hex')}`,
+        refreshedAt: new Date().toISOString(),
+        lastError: null,
+      })
+    }
+    const alreadyBumped = new Set<string>()
+    for (const id of changed) repos.bumpAgentsForList(tx, id, alreadyBumped)
+  })
+  return changed
 }
 
 /** Rebuild ip_list_entries from config.items (manual lists). */
-export async function rebuildManualListEntries(
+export async function rebuildManualListEntries(db: Db, listId: string, _visiting: Set<string> = new Set()): Promise<string[]> {
-  db: Db,
+  await publishListCascade(db, listId)
-  listId: string,
+  return repos.listIpListEntries(db, listId).map((entry) => entry.cidr)
-  visiting: Set<string> = new Set(),
-): Promise<string[]> {
-  const list = repos.getIpList(db, listId)
-  if (!list || !isManualListType(list.type)) return []
-
-  if (visiting.has(listId)) {
-    throw new Error(`Циклическая ссылка списков: ${listId}`)
-  }
-  visiting.add(listId)
-
-  try {
-    const config = getListConfig(list)
-    const items = readManualItems(list.configJson)
-    const nextItems: ListConfigItem[] = []
-    const all: string[] = []
-
-    for (const item of items) {
-      try {
-        const cidrs = await expandItem(
-          db,
-          { ...item, resolved_cidrs: undefined },
-          visiting,
-        )
-        nextItems.push({ ...item, resolved_cidrs: cidrs })
-        all.push(...cidrs)
-      } catch {
-        nextItems.push({ ...item, resolved_cidrs: item.resolved_cidrs ?? [] })
-        if (item.resolved_cidrs?.length) all.push(...item.resolved_cidrs)
-      }
-    }
-
-    config.items = nextItems
-    delete config.domains
-    repos.updateIpList(db, listId, { configJson: JSON.stringify(config) })
-
-    const cidrs = uniqCidrs(all)
-    repos.replaceIpListEntries(db, listId, cidrs)
-    return cidrs
-  } finally {
-    visiting.delete(listId)
-  }
 }
 
 /** Rebuild list and all ancestor lists that nest it; bump agents. */
-export async function rebuildListCascade(
+export async function rebuildListCascade(db: Db, listId: string): Promise<string[]> {
-  db: Db,
+  return publishListCascade(db, listId)
-  listId: string,
-): Promise<string[]> {
-  const affected: string[] = []
-  const queue = [listId]
-  const seen = new Set<string>()
-
-  while (queue.length) {
-    const id = queue.shift()!
-    if (seen.has(id)) continue
-    seen.add(id)
-    const list = repos.getIpList(db, id)
-    if (!list) continue
-    if (isManualListType(list.type)) {
-      await rebuildManualListEntries(db, id)
-    }
-    affected.push(id)
-    for (const parentId of findParentListIds(db, id)) {
-      if (!seen.has(parentId)) queue.push(parentId)
-    }
-  }
-
-  for (const id of affected) {
-    repos.bumpAgentsForList(db, id)
-  }
-  return affected
 }
 
 function normalizeInputItem(input: ListEntryInput): ListConfigItem {
@@ -170,16 +170,17 @@ function normalizeInputItem(input: ListEntryInput): ListConfigItem {
     return { kind: 'list', value: input.value.trim() }
   }
   if (input.kind === 'hostname') {
-    return { kind: 'hostname', value: input.value.trim().toLowerCase() }
+    return { kind: 'hostname', value: canonicalHostname(input.value) }
   }
-  return { kind: input.kind, value: input.value.trim() }
+  return { kind: input.kind, value: input.kind === 'ip' ? canonicalIp(input.value) : canonicalCidr(input.value) }
 }
 
-export async function addListEntries(
+async function mutateListEntries(
   db: Db,
   listId: string,
   input: { values?: string[]; items?: ListEntryInput[] },
-): Promise<{ items: ListConfigItem[]; entries: string[] }> {
+  allowNoop: boolean,
+): Promise<ListImportSummary> {
   const list = repos.getIpList(db, listId)
   if (!list) throw new Error('List not found')
   if (!isManualListType(list.type)) {
@@ -218,14 +219,14 @@ export async function addListEntries(
     const key = `${parsed.kind}:${parsed.value.toLowerCase()}`
     if (existing.has(key)) continue
 
-    const resolved = await expandItem(db, parsed, new Set([listId]))
+    const item: ListConfigItem = parsed
-    const item: ListConfigItem = { ...parsed, resolved_cidrs: resolved }
     items.push(item)
     existing.add(key)
     added.push(item)
   }
 
   if (added.length === 0) {
+    if (allowNoop) return { ...repos.getIpListEntrySummary(db, listId)!, changed: false }
     throw new Error('Нет новых записей для добавления')
   }
 
@@ -233,16 +234,17 @@ export async function addListEntries(
   delete config.domains
 
   const type = list.type === 'domains' ? 'static' : list.type
-  repos.updateIpList(db, listId, {
+  const changed = await publishListCascade(db, listId, { type, configJson: JSON.stringify(config) })
-    type,
-    configJson: JSON.stringify(config),
-  })
 
-  await rebuildListCascade(db, listId)
+  return { ...repos.getIpListEntrySummary(db, listId)!, changed: changed.includes(listId) }
-  repos.updateIpList(db, listId, {
+}
-    refreshedAt: new Date().toISOString(),
+
-    lastError: null,
+export async function addListEntriesSummary(db: Db, listId: string, input: { values?: string[]; items?: ListEntryInput[] }): Promise<ListImportSummary> {
-  })
+  return mutateListEntries(db, listId, input, true)
+}
+
+export async function addListEntries(db: Db, listId: string, input: { values?: string[]; items?: ListEntryInput[] }): Promise<{ items: ListConfigItem[]; entries: string[] }> {
+  await mutateListEntries(db, listId, input, false)
 
   return {
     items: readManualItems(repos.getIpList(db, listId)!.configJson),
@@ -250,11 +252,11 @@ export async function addListEntries(
   }
 }
 
-export async function deleteListEntry(
+async function mutateDeleteListEntry(
   db: Db,
   listId: string,
   value: string,
-): Promise<{ items: ListConfigItem[]; entries: string[] }> {
+): Promise<ListImportSummary> {
   const list = repos.getIpList(db, listId)
   if (!list) throw new Error('List not found')
   if (!isManualListType(list.type)) {
@@ -274,16 +276,17 @@ export async function deleteListEntry(
   delete config.domains
 
   const type = list.type === 'domains' ? 'static' : list.type
-  repos.updateIpList(db, listId, {
+  const changed = await publishListCascade(db, listId, { type, configJson: JSON.stringify(config) })
-    type,
-    configJson: JSON.stringify(config),
-  })
 
-  await rebuildListCascade(db, listId)
+  return { ...repos.getIpListEntrySummary(db, listId)!, changed: changed.includes(listId) }
-  repos.updateIpList(db, listId, {
+}
-    refreshedAt: new Date().toISOString(),
+
-    lastError: null,
+export async function deleteListEntrySummary(db: Db, listId: string, value: string): Promise<ListImportSummary> {
-  })
+  return mutateDeleteListEntry(db, listId, value)
+}
+
+export async function deleteListEntry(db: Db, listId: string, value: string): Promise<{ items: ListConfigItem[]; entries: string[] }> {
+  await mutateDeleteListEntry(db, listId, value)
 
   return {
     items: readManualItems(repos.getIpList(db, listId)!.configJson),
@@ -291,6 +294,19 @@ export async function deleteListEntry(
   }
 }
 
+export function mapListDetailPage(db: Db, listId: string, query: ListDetailPageQuery) {
+  const page = repos.getListDetailPage(db, listId, query)
+  if (!page) return null
+  const items = page.rows.map((row) => {
+    let resolved = JSON.parse(row.resolved_json) as string[]
+    if (resolved.length === 0 && (row.kind === 'ip' || row.kind === 'cidr')) resolved = normalizeItemToCidrs(row)
+    const count = row.stored_count || resolved.length
+    return { kind: row.kind, value: row.value, list_name: row.list_name, resolved_count: count, resolved_cidrs: resolved, resolved_truncated: count > resolved.length }
+  })
+  return { ...page.metadata, config_json: '{}', total: page.total, limit: query.limit, offset: query.offset,
+    items, entries: items.flatMap((item) => item.resolved_cidrs) }
+}
+
 export function mapListDetail(db: Db, listId: string) {
   const l = repos.getIpList(db, listId)
   if (!l) return null

apps/api/src/services/lists/refresh.ts

@@ -1,51 +1,47 @@
-import { createHash } from 'node:crypto'
 import type { Db } from '@evofw/db'
 import { repos } from '@evofw/db'
 import { refreshAllHostnameRules } from '../policy/resolve-hostname.js'
-import {
+import { rebuildListCascade, publishListCascade } from './entries.js'
-  findParentListIds,
-  rebuildListCascade,
-  rebuildManualListEntries,
-} from './entries.js'
 import { uniqCidrs } from '../uniq.js'
 import { decryptSecret } from '../secret-cipher.js'
+import { fetchEvobgpPages } from '../evobgp.js'
+import { safeFetchJson, SOURCE_LIMITS } from '../safe-fetch.js'
+
 
-function hashCidrs(cidrs: string[]): string {
-  return `sha256:${createHash('sha256').update(cidrs.join('\n')).digest('hex')}`
-}
 
 async function fetchJsonUrl(url: string): Promise<string[]> {
-  const res = await fetch(url, {
+  const { data } = await safeFetchJson(url)
-    headers: { Accept: 'application/json' },
-    signal: AbortSignal.timeout(30_000),
-  })
-  if (!res.ok) throw new Error(`JSON URL HTTP ${res.status}`)
-  const data = (await res.json()) as unknown
   const out: string[] = []
+  let recognized = false
   const push = (v: unknown) => {
-    if (typeof v === 'string' && v.trim()) out.push(v.trim())
+    if (out.length >= SOURCE_LIMITS.elements) throw new Error('JSON prefix element limit exceeded')
+    if (typeof v !== 'string' || !v.trim()) throw new Error('Invalid JSON prefix item')
+    out.push(v.trim())
   }
   if (Array.isArray(data)) {
+    recognized = true
     for (const item of data) {
       if (typeof item === 'string') push(item)
       else if (item && typeof item === 'object') {
         const o = item as Record<string, unknown>
         push(o.cidr ?? o.prefix ?? o.ip ?? o.network)
-      }
+      } else throw new Error('Invalid JSON prefix item')
     }
   } else if (data && typeof data === 'object') {
     const o = data as Record<string, unknown>
     const arr = (o.prefixes ?? o.cidrs ?? o.ips ?? o.items) as unknown
     if (Array.isArray(arr)) {
+      recognized = true
       for (const item of arr) {
         if (typeof item === 'string') push(item)
         else if (item && typeof item === 'object') {
           const x = item as Record<string, unknown>
           push(x.cidr ?? x.prefix ?? x.ip)
+        } else throw new Error('Invalid JSON prefix item')
       }
     }
   }
-  }
+  if (!recognized) throw new Error('Invalid JSON prefix response')
   return uniqCidrs(out)
 }
 
@@ -62,20 +58,12 @@ async function resolveEvobgpCommunityId(
   if (UUID_RE.test(key)) return key
 
   // Heal lists created when Autocomplete stored label instead of UUID.
-  const res = await fetch(`${base}/v1/communities?limit=200`, {
+  const pages = await fetchEvobgpPages(base, token, '/v1/communities', 200)
-    headers: {
+  const items = pages.flatMap((page) => {
-      Authorization: `Bearer ${token}`,
+    if (!Array.isArray(page.items)) throw new Error('Invalid EvoBGP communities')
-      Accept: 'application/json',
+    return page.items as { id?: string; community?: string; title?: string | null }[]
-    },
-    signal: AbortSignal.timeout(20_000),
   })
-  if (!res.ok) {
+  const hit = items.find((c) => {
-    throw new Error(`EvoBGP communities HTTP ${res.status}`)
-  }
-  const data = (await res.json()) as {
-    items?: { id?: string; community?: string; title?: string | null }[]
-  }
-  const hit = (data.items ?? []).find((c) => {
     if (!c.id || !c.community) return false
     if (c.id === key || c.community === key) return true
     if (c.title && `${c.community} · ${c.title}` === key) return true
@@ -87,40 +75,50 @@ async function resolveEvobgpCommunityId(
   return hit.id
 }
 
-async function fetchEvobgpCommunity(
+async function fetchEvobgpCommunity(apiUrl: string, token: string, communityId: string): Promise<{ cidrs: string[]; resolvedId: string }> {
-  apiUrl: string,
-  token: string,
-  communityId: string,
-): Promise<{ cidrs: string[]; resolvedId: string }> {
   const base = apiUrl.replace(/\/$/, '')
   const resolvedId = await resolveEvobgpCommunityId(base, token, communityId)
-  const url = `${base}/v1/communities/${encodeURIComponent(resolvedId)}/prefixes?limit=5000`
+  const pages = await fetchEvobgpPages(base, token, `/v1/communities/${encodeURIComponent(resolvedId)}/prefixes`, 5000)
-  const res = await fetch(url, {
+  const cidrs = pages.flatMap((page) => {
-    headers: {
+    if (Array.isArray(page.prefixes)) return page.prefixes as string[]
-      Authorization: `Bearer ${token}`,
+    if (Array.isArray(page.items)) return page.items.map((item) => {
-      Accept: 'application/json',
+      if (!item || typeof item !== 'object' || typeof item.prefix !== 'string') throw new Error('Invalid EvoBGP prefix item')
-    },
+      return item.prefix as string
-    signal: AbortSignal.timeout(45_000),
     })
-  if (!res.ok) {
+    throw new Error('Invalid EvoBGP prefix page')
-    throw new Error(`EvoBGP community prefixes HTTP ${res.status}`)
+  })
-  }
+  return { cidrs: uniqCidrs(cidrs), resolvedId }
-  const data = (await res.json()) as {
-    items?: { prefix?: string }[]
-    prefixes?: string[]
-  }
-  let cidrs: string[] = []
-  if (Array.isArray(data.prefixes) && data.prefixes.length > 0) {
-    cidrs = uniqCidrs(data.prefixes)
-  } else if (Array.isArray(data.items)) {
-    cidrs = uniqCidrs(data.items.map((i) => i.prefix ?? '').filter(Boolean))
-  }
-  return { cidrs, resolvedId }
 }
 
-export async function refreshIpList(db: Db, listId: string): Promise<void> {
+let activeRefreshes = 0
+const waiting: (() => void)[] = []
+const refreshing = new WeakMap<Db, Map<string, Promise<void>>>()
+
+async function withRefreshSlot(work: () => Promise<void>): Promise<void> {
+  if (activeRefreshes >= 2) await new Promise<void>(resolve => waiting.push(resolve))
+  else activeRefreshes++
+  try { await work() } finally {
+    const next = waiting.shift()
+    if (next) next()
+    else activeRefreshes--
+  }
+}
+
+/** Coalesce duplicate triggers per DB/list; at most two refreshes process-wide. */
+export function refreshIpList(db: Db, listId: string): Promise<void> {
+  let running = refreshing.get(db)
+  if (!running) { running = new Map(); refreshing.set(db, running) }
+  const existing = running.get(listId)
+  if (existing) return existing
+  const task = withRefreshSlot(() => performRefresh(db, listId)).finally(() => running!.delete(listId))
+  running.set(listId, task)
+  return task
+}
+
+async function performRefresh(db: Db, listId: string): Promise<void> {
   const list = repos.getIpList(db, listId)
   if (!list) return
+  const sourceSettings = [repos.getSetting(db, 'evobgp_api_url'), repos.getSetting(db, 'evobgp_api_token')]
 
   let config: Record<string, unknown> = {}
   try {
@@ -132,12 +130,14 @@ export async function refreshIpList(db: Db, listId: string): Promise<void> {
   try {
     let cidrs: string[] = []
     if (list.type === 'static' || list.type === 'domains') {
-      cidrs = await rebuildManualListEntries(db, listId)
+      await rebuildListCascade(db, listId)
+      repos.updateIpList(db, listId, { refreshedAt: new Date().toISOString(), lastError: null })
+      return
     } else if (list.type === 'json_url') {
       const url = String(config.url ?? '')
       if (!url) throw new Error('config.url required')
       cidrs = await fetchJsonUrl(url)
-      repos.replaceIpListEntries(db, listId, cidrs)
+
     } else if (list.type === 'evobgp_community') {
       const apiUrl =
         String(config.api_url ?? '') ||
@@ -152,39 +152,23 @@ export async function refreshIpList(db: Db, listId: string): Promise<void> {
       }
       const fetched = await fetchEvobgpCommunity(apiUrl, token, communityId)
       cidrs = fetched.cidrs
-      repos.replaceIpListEntries(db, listId, cidrs)
+
       // Persist resolved UUID if list was saved with autocomplete label.
       if (fetched.resolvedId !== communityId) {
-        repos.updateIpList(db, listId, {
+        config.community_id = fetched.resolvedId
-          configJson: JSON.stringify({
-            ...config,
-            community_id: fetched.resolvedId,
-          }),
-        })
-      }
       }
+    } else throw new Error('Unsupported list source type')
 
-    const contentHash = hashCidrs(cidrs)
+    if (!cidrs.length && config.allow_empty !== true) throw new Error('Empty source snapshot quarantined; set config.allow_empty=true to publish it')
-    repos.updateIpList(db, listId, {
+    if (list.type === 'evobgp_community' && sourceSettings.some((value, index) => value !== repos.getSetting(db, index === 0 ? 'evobgp_api_url' : 'evobgp_api_token'))) {
-      contentHash,
+      throw new Error('EvoBGP settings changed during refresh; retry refresh')
-      refreshedAt: new Date().toISOString(),
-      lastError: null,
-    })
-
-    // Cascade to parents that nest this list (skip self rebuild for non-manual —
-    // CIDRs already replaced above).
-    if (list.type === 'static' || list.type === 'domains') {
-      await rebuildListCascade(db, listId)
-    } else {
-      repos.bumpAgentsForList(db, listId)
-      for (const parentId of findParentListIds(db, listId)) {
-        await rebuildListCascade(db, parentId)
-      }
     }
+    await publishListCascade(db, listId, { cidrs, configJson: JSON.stringify(config), expected: list })
   } catch (err) {
+    const current = repos.getIpList(db, listId)
+    if (!current || current.configJson !== list.configJson || current.type !== list.type || current.contentHash !== list.contentHash || current.updatedAt !== list.updatedAt) return
     repos.updateIpList(db, listId, {
-      lastError: err instanceof Error ? err.message : String(err),
+      lastError: (err instanceof Error ? err.message : String(err)).slice(0, 512),
-      refreshedAt: new Date().toISOString(),
     })
   }
 }

apps/api/src/services/operational.test.ts

@@ -0,0 +1,187 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { createHash } from 'node:crypto'
+import { mkdtempSync, writeFileSync, rmSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import { join } from 'node:path'
+import { repos, createMemoryDb, runMigrations, appendAudit } from '@evofw/db'
+import { buildApp } from '../app.js'
+import { loadConfig, trustedProxyCidrs } from '../config.js'
+import { IdentityLimiter, routeSecurityFor } from '../plugins/operational.js'
+import { AuditOutbox } from './audit-outbox.js'
+import { ingestAuditEventSchema } from '@evofw/shared'
+const opened: Awaited<ReturnType<typeof buildApp>>[] = []
+const temporary: string[] = []
+afterEach(async () => {
+  for (const app of opened.splice(0)) await app.close()
+  for (const dir of temporary.splice(0)) rmSync(dir, { recursive: true, force: true })
+  vi.restoreAllMocks(); vi.unstubAllEnvs()
+})
+async function application(extra = {}) {
+  const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: true, staticDir: null, authAuditIngestSecret: null, logLevel: 'silent', ...extra } })
+  opened.push(app); await app.ready(); return app
+}
+const userToken = (app: Awaited<ReturnType<typeof application>>, id = 'operator', permissions = ['fw:settings:admin']) => app.jwt.sign({ sub: id, apps: ['fw'], permissions, iss: loadConfig().authIssuer })
+
+describe('operational request guards', () => {
+  it('covers every protected route, fails unknown mappings and preserves the filtered-summary exception', async () => {
+    const app = await application()
+    const routes = app.routeSecurity.filter(route => route.url.startsWith('/api/v1/'))
+    expect(routes.length).toBeGreaterThan(30)
+    for (const route of routes) {
+      expect(route.policy === 'protected' ? route.permission : route.url === '/api/v1/ui/summary' || route.url === '/api/v1/auth/config').toBeTruthy()
+    }
+    expect(routeSecurityFor('GET', '/api/v1/ui/summary').policy).toBe('section-filtered')
+    expect(routeSecurityFor('GET', '/api/v1/agents/:id/stats/raw').permission).toBe('fw:stats:read')
+    expect(routeSecurityFor('GET', '/api/v1/stats/export.csv').permission).toBe('fw:stats:read')
+    expect(routeSecurityFor('GET', '/api/v1/agents/:id/explain').permission).toBe('fw:agents:read')
+    expect(() => routeSecurityFor('GET', '/api/v1/unmapped-feature')).toThrow('lacks permission')
+    expect(() => routeSecurityFor('GET', '/v1/agent/unmapped-feature')).toThrow('authentication mapping')
+    const token = userToken(app, 'limited', ['fw:lists:read'])
+    const summary = await app.inject({ url: '/api/v1/ui/summary', headers: { authorization: `Bearer ${token}` } })
+    expect(summary.statusCode).toBe(200); expect(summary.json().lists).toBeDefined(); expect(summary.json().agents).toBeUndefined()
+    expect((await app.inject({ url: '/api/v1/diagnostics', headers: { authorization: `Bearer ${token}` } })).statusCode).toBe(403)
+  })
+  it('accepts 100 authenticated agents with reports, exact retries and concurrent summary reads behind one NAT', async () => {
+    const app = await application()
+    for (let i = 0; i < 100; i++) repos.insertAgent(app.db, { id: `nat-${i}`, name: `nat-${i}`, tokenPrefix: `nat-${i}`, tokenHash: createHash('sha256').update(`token-${i}`).digest('hex'), status: 'approved' })
+    const statuses: number[] = []
+    const token = userToken(app, 'reader', ['fw:stats:read'])
+    let next = 0
+    await Promise.all(Array.from({ length: 10 }, async () => {
+      while (next < 100) {
+        const i = next++
+        const headers = { authorization: `Bearer token-${i}` }
+        const remoteAddress = '198.51.100.44'
+        for (let sequence = 1; sequence <= 3; sequence++) {
+          const payload = { telemetry_version: 2, report_id: `nat-report-${sequence}`, sequence, boot_id: 'boot', observed_at: new Date(Date.now() - (4 - sequence) * 60_000).toISOString(), interval_ms: 60_000, counter_epoch: { traffic: 'traffic' }, capabilities: ['policy_ipv4'], quality: { partial: false, truncated: false, unsupported: [], lost_reports: 0 }, status: 'ok', packets_dropped: sequence * 100, packets_accepted: sequence * 200, detail_counts: { ip_found: 0, ip_sent: 0, port_found: 0, port_sent: 0 } }
+          for (let retry = 0; retry < 2; retry++) statuses.push((await app.inject({ method: 'POST', url: '/v1/agent/apply-report', headers, payload, remoteAddress })).statusCode)
+        }
+        statuses.push((await app.inject({ url: '/v1/agent/policy', headers, remoteAddress })).statusCode)
+        statuses.push((await app.inject({ url: '/api/v1/ui/summary', headers: { authorization: `Bearer ${token}` }, remoteAddress })).statusCode)
+      }
+    }))
+    expect(statuses).toHaveLength(800); expect(new Set(statuses)).toEqual(new Set([200]))
+    for (let i = 0; i < 100; i++) expect(repos.getAgent(app.db, `nat-${i}`)!.totalPacketsDropped).toBe(200)
+  }, 30_000)
+  it('bounds identity buckets and keeps independent identities/enrollment limits', async () => {
+    const limiter = new IdentityLimiter(2)
+    expect(limiter.take('a', 1, 0)).toBe(true); expect(limiter.take('a', 1, 1)).toBe(false)
+    expect(limiter.take('b', 1, 1)).toBe(true); expect(limiter.take('c', 1, 1)).toBe(false)
+    expect(limiter.take('c', 1, 60_001)).toBe(true); expect(limiter.stats().buckets).toBeLessThanOrEqual(2)
+    const app = await application()
+    for (let i = 0; i < 20; i++) expect((await app.inject({ method: 'POST', url: '/v1/agent/enroll', payload: {}, remoteAddress: '198.51.100.44' })).statusCode).not.toBe(429)
+    expect((await app.inject({ method: 'POST', url: '/v1/agent/enroll', payload: {}, remoteAddress: '198.51.100.44' })).statusCode).toBe(429)
+  })
+  it('uses explicit proxy trust for audit IP and never includes settings values in local/outbox audit', async () => {
+    expect(trustedProxyCidrs(undefined)).toBe(false)
+    expect(() => trustedProxyCidrs('true')).toThrow('explicit')
+    for (const trustProxy of [false, ['127.0.0.1/32']] as const) {
+      const app = await application({ trustProxy, authAuditIngestSecret: 'portal-ingest-secret' })
+      const token = userToken(app)
+      const response = await app.inject({ method: 'PUT', url: '/api/v1/settings', headers: { authorization: `Bearer ${token}`, 'x-forwarded-for': '203.0.113.8' }, remoteAddress: '127.0.0.1', payload: { evobgp_api_token: 'NEVER-AUDIT-THIS-SECRET' } })
+      expect(response.statusCode).toBe(200)
+      const audit = app.sqlite.prepare("SELECT details_json,ip FROM audit_log WHERE action='settings.update'").get() as { details_json: string; ip: string }
+      expect(audit.ip).toBe(trustProxy === false ? '127.0.0.1' : '203.0.113.8')
+      expect(JSON.parse(audit.details_json)).toEqual({ changed_keys: ['evobgp_api_token'] })
+      const queued = app.sqlite.prepare('SELECT payload_json FROM audit_outbox').get() as { payload_json: string }
+      expect(queued.payload_json).not.toContain('NEVER-AUDIT-THIS-SECRET'); expect(queued.payload_json).not.toContain('portal-ingest-secret')
+    }
+  })
+  it('rate-limits failed authentication while a valid agent behind the same NAT remains usable', async () => {
+    const app = await application()
+    repos.insertAgent(app.db, { id: 'valid', name: 'valid', tokenPrefix: 'valid', tokenHash: createHash('sha256').update('valid-token').digest('hex'), status: 'approved' })
+    const remoteAddress = '198.51.100.44'
+    for (let i = 0; i < 60; i++) expect((await app.inject({ url: '/v1/agent/policy', remoteAddress, headers: { authorization: 'Bearer invalid' } })).statusCode).toBe(401)
+    expect((await app.inject({ url: '/v1/agent/policy', remoteAddress, headers: { authorization: 'Bearer invalid' } })).statusCode).toBe(429)
+    expect((await app.inject({ url: '/v1/agent/policy', remoteAddress, headers: { authorization: 'Bearer valid-token' } })).statusCode).toBe(200)
+  })
+  it('serves JSON 404 for unknown API paths when SPA fallback is enabled', async () => {
+    const dir = mkdtempSync(join(tmpdir(), 'evofw-spa-')); temporary.push(dir); writeFileSync(join(dir, 'index.html'), '<html>SPA</html>')
+    const app = await application({ staticDir: dir })
+    const headers = { authorization: `Bearer ${userToken(app)}` }
+    for (const url of ['/api/v1/not-a-route', '/v1/agent/not-a-route', '/api']) {
+      const response = await app.inject({ url, headers }); expect(response.statusCode).toBe(404); expect(response.json().error.code).toBe('NOT_FOUND')
+    }
+    expect((await app.inject({ url: '/agents', headers })).body).toContain('SPA')
+  })
+})
+
+describe('durable audit delivery', () => {
+  it('persists retries across worker recreation and acknowledges only successful batches', async () => {
+    const { sqlite } = createMemoryDb(); runMigrations(sqlite)
+    const config = { ...loadConfig(), authAuditIngestSecret: 'secret', authPortalUrl: 'https://portal.test' }
+    const first = new AuditOutbox(sqlite, config)
+    const event = { event_id: 'event-one', source_app: 'fw', action: 'test', severity: 'info', summary: 'test', created_at: new Date().toISOString() } as const
+    try {
+      first.enqueue(event)
+      const fetcher = vi.spyOn(globalThis, 'fetch').mockResolvedValue(new Response('', { status: 503 }))
+      await first.flush(1000)
+      expect(first.size()).toBe(1)
+      const row = sqlite.prepare('SELECT attempts,next_attempt_ms FROM audit_outbox').get() as { attempts: number; next_attempt_ms: number }
+      expect(row.attempts).toBe(1); expect(row.next_attempt_ms).toBe(6000)
+      await first.close()
+      const restarted = new AuditOutbox(sqlite, config)
+      await restarted.flush(5999); expect(fetcher).toHaveBeenCalledTimes(1)
+      fetcher.mockResolvedValue(new Response('', { status: 200 }))
+      await restarted.flush(6000); expect(restarted.size()).toBe(0)
+      const sent = JSON.parse(fetcher.mock.calls[1]![1]!.body as string)
+      expect(sent.events[0].event_id).toBe('event-one')
+      await restarted.close()
+    } finally { sqlite.close() }
+  })
+  it('aborts an active worker on shutdown and retains queued events', async () => {
+    const { sqlite } = createMemoryDb(); runMigrations(sqlite)
+    const queue = new AuditOutbox(sqlite, { ...loadConfig(), authAuditIngestSecret: 'secret' })
+    queue.enqueue({ event_id: 'active', source_app: 'fw', action: 'test', severity: 'info', summary: 'test', created_at: new Date().toISOString() })
+    vi.spyOn(globalThis, 'fetch').mockImplementation(async (_url, options) => new Promise((_, reject) => options!.signal!.addEventListener('abort', () => reject(new Error('aborted')))))
+    const pending = queue.flush()
+    await queue.close(); await pending
+    expect(queue.size()).toBe(1)
+    sqlite.close()
+  })
+  it('rejects queue/payload exhaustion before unbounded allocation and exposes bounded backpressure', async () => {
+    const app = await application({ authAuditIngestSecret: 'secret' })
+    const insert = app.sqlite.prepare('INSERT INTO audit_outbox(event_id,payload_json,created_at) VALUES (?,?,?)')
+    app.sqlite.transaction(() => { for (let i = 0; i < 10_000; i++) insert.run(`full-${i}`, '{}', new Date().toISOString()) })()
+    const settingBefore = repos.getSetting(app.db, 'show_quick_actions')
+    const response = await app.inject({ method: 'PUT', url: '/api/v1/settings', headers: { authorization: `Bearer ${userToken(app)}` }, payload: { show_quick_actions: 'false' } })
+    expect(response.statusCode).toBe(503)
+    expect(repos.getSetting(app.db, 'show_quick_actions')).toBe(settingBefore)
+    expect(app.auditOutbox.stats().pending).toBe(10_000)
+    expect(() => app.auditOutbox.enqueue({ event_id: 'overflow', source_app: 'fw', action: 'test', severity: 'info', summary: 'test', details: { data: 'x'.repeat(17_000) }, created_at: new Date().toISOString() })).toThrow('byte limit')
+  })
+  it('normalizes a real long-name mutation and dev actor to the portal contract', async () => {
+    const app = await application({ authRequired: false, authAuditIngestSecret: 'secret' })
+    const response = await app.inject({ method: 'POST', url: '/api/v1/lists', payload: { name: 'Long name '.repeat(70), type: 'static' } })
+    expect(response.statusCode).toBe(200)
+    const row = app.sqlite.prepare('SELECT payload_json FROM audit_outbox').get() as { payload_json: string }
+    const event = JSON.parse(row.payload_json)
+    expect(ingestAuditEventSchema.safeParse(event).success).toBe(true)
+    expect(event.summary.length).toBeLessThanOrEqual(500); expect(event.summary.length).toBeGreaterThan(480); expect(event.actor_email).toBeNull()
+    expect(app.sqlite.prepare('SELECT count(*) AS n FROM audit_log').get()).toEqual({ n: 1 })
+  })
+  it('quarantines stored malformed rows, repairs known legacy fields, and delivers healthy rows without poisoning', async () => {
+    const app = await application({ authAuditIngestSecret: 'secret' })
+    const insert = app.sqlite.prepare('INSERT INTO audit_outbox(event_id,payload_json,created_at) VALUES (?,?,?)')
+    const created_at = new Date().toISOString()
+    insert.run('malformed', 'not JSON private-content', created_at)
+    appendAudit(app.db, { eventId: 'malformed', action: 'test', summary: 'Retained local audit event' })
+    insert.run('legacy', JSON.stringify({ event_id: 'legacy', source_app: 'fw', action: 'test', summary: 'x'.repeat(512), actor_email: 'dev@local', created_at }), created_at)
+    app.auditOutbox.enqueue({ event_id: 'healthy', source_app: 'fw', action: 'test', summary: 'valid', created_at })
+    const fetcher = vi.spyOn(globalThis, 'fetch').mockImplementation(async (_url, options) => {
+      const { events } = JSON.parse(options!.body as string)
+      for (const event of events) expect(ingestAuditEventSchema.safeParse(event).success).toBe(true)
+      expect(events.map((event: { event_id: string }) => event.event_id)).toEqual(['legacy', 'healthy'])
+      return new Response('', { status: 200 })
+    })
+    await app.auditOutbox.flush()
+    expect(fetcher).toHaveBeenCalledTimes(1)
+    expect(app.auditOutbox.stats()).toMatchObject({ pending: 0, quarantined: 1 })
+    expect(JSON.stringify(app.auditOutbox.stats())).not.toContain('private-content')
+    for (let i = 0; i < 110; i++) insert.run(`invalid-${i}`, '{}', created_at)
+    for (let i = 0; i < 5; i++) await app.auditOutbox.flush()
+    expect(app.auditOutbox.stats().quarantined).toBe(100)
+    expect(fetcher).toHaveBeenCalledTimes(1)
+    expect(app.sqlite.prepare('SELECT summary FROM audit_log WHERE event_id=?').get('malformed')).toEqual({ summary: 'Retained local audit event' })
+  })
+})

apps/api/src/services/policy/cache-approval.test.ts

@@ -0,0 +1,121 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { buildApp } from '../../app.js'
+import { loadConfig } from '../../config.js'
+import { hashToken } from '../../plugins/auth.js'
+import { evaluateAgentPolicy } from './evaluate.js'
+import { policyCacheFor } from './cache.js'
+import { refreshAllHostnameRules } from './resolve-hostname.js'
+
+const dns = vi.hoisted(() => ({ resolve4: vi.fn(), resolve6: vi.fn(), cancel: vi.fn() }))
+vi.mock('node:dns/promises', async (original) => ({ ...await original<typeof import('node:dns/promises')>(), Resolver: class { resolve4 = dns.resolve4; resolve6 = dns.resolve6; cancel = dns.cancel } }))
+const opened: ReturnType<typeof createMemoryDb>[] = []
+const apps: Awaited<ReturnType<typeof buildApp>>[] = []
+afterEach(async () => { for (const app of apps.splice(0)) await app.close(); for (const mem of opened.splice(0)) mem.sqlite.close(); vi.resetAllMocks() })
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  repos.insertAgent(mem.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: 'a', status: 'approved' })
+  return mem
+}
+function hostnameFixture() {
+  const mem = memory(); repos.ensureSharedSetAssigned(mem.db, 'a')
+  for (const [id, hostname, cidr, priority] of [['h1', 'first.test', '192.0.2.1/32', 10], ['h2', 'second.test', '192.0.2.2/32', 20]] as const) {
+    repos.insertPolicyRule(mem.db, { id, setId: 'set-shared-default', hostname, action: 'deny', priority })
+    repos.replaceResolvedForRule(mem.db, id, [cidr])
+  }
+  dns.resolve6.mockResolvedValue([])
+  return mem
+}
+
+describe('cache assignment and DNS publication invalidation', () => {
+  it('refreshes cached preview set names after a rename without bumping again for an identical name', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } }); apps.push(app); await app.ready()
+    repos.insertAgent(app.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: 'a', status: 'approved' })
+    repos.ensureSharedSetAssigned(app.db, 'a')
+    repos.insertPolicyRule(app.db, { id: 'deny', setId: 'set-shared-default', action: 'deny', cidr: '10.0.0.0/8', priority: 10 })
+    const before = (await app.inject({ url: '/api/v1/agents/a/preview' })).json()
+    const renamed = await app.inject({ method: 'PATCH', url: '/api/v1/policy-sets/set-shared-default', payload: { name: 'Renamed source set' } })
+    expect(renamed.statusCode).toBe(200)
+    const after = (await app.inject({ url: '/api/v1/agents/a/preview' })).json()
+    expect(after.chain[0].set_name).toBe('Renamed source set')
+    expect(after.generation).toBe(before.generation + 1)
+    expect(after.hash).toBe(before.hash)
+    await app.inject({ method: 'PATCH', url: '/api/v1/policy-sets/set-shared-default', payload: { name: 'Renamed source set' } })
+    expect((await app.inject({ url: '/api/v1/agents/a/preview' })).json().generation).toBe(after.generation)
+  })
+
+  it('invalidates a preview cached before single and bulk approval, with exactly one insertion bump and idempotent retries', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } }); apps.push(app); await app.ready()
+    repos.insertPolicyRule(app.db, { id: 'deny', setId: 'set-shared-default', action: 'deny', cidr: '10.0.0.0/8', priority: 10 })
+    for (const id of ['single', 'bulk']) {
+      repos.insertAgent(app.db, { id, name: id, tokenPrefix: id, tokenHash: hashToken(`token-${id}`), status: 'pending' })
+      const preview = await app.inject({ url: `/api/v1/agents/${id}/preview` })
+      expect(preview.statusCode).toBe(200)
+      expect(preview.json().summary.cidrs_deny).toBe(0)
+      expect(preview.json().generation).toBe(1)
+    }
+    const single = await app.inject({ method: 'POST', url: '/api/v1/agents/single/approve' })
+    expect(single.statusCode).toBe(200)
+    expect(single.json().policy_generation).toBe(2)
+    const bulk = await app.inject({ method: 'POST', url: '/api/v1/agents/approve-bulk', payload: { agent_ids: ['bulk'] } })
+    expect(bulk.statusCode).toBe(200)
+    expect(bulk.json().items[0].policy_generation).toBe(2)
+    for (const id of ['single', 'bulk']) {
+      const policy = await app.inject({ url: '/v1/agent/policy', headers: { authorization: `Bearer token-${id}` } })
+      expect(policy.statusCode).toBe(200)
+      expect(policy.json()).toMatchObject({ generation: 2, deny_cidrs: ['10.0.0.0/8'] })
+      expect(repos.ensureSharedSetAssigned(app.db, id)).toBe(false)
+      expect(repos.getAgent(app.db, id)!.policyGeneration).toBe(2)
+    }
+    expect((await app.inject({ method: 'POST', url: '/api/v1/agents/single/approve' })).json().policy_generation).toBe(2)
+    expect((await app.inject({ method: 'POST', url: '/api/v1/agents/approve-bulk', payload: { agent_ids: ['bulk'] } })).json().items).toEqual([])
+    expect(repos.getAgent(app.db, 'bulk')!.policyGeneration).toBe(2)
+  })
+
+  it('rolls back assignment insertion when the generation bump fails, and existing assignments never bump again', () => {
+    const mem = memory()
+    mem.sqlite.exec("CREATE TRIGGER reject_generation BEFORE UPDATE OF policy_generation ON agents BEGIN SELECT RAISE(ABORT,'test generation failure'); END")
+    expect(() => repos.ensureSharedSetAssigned(mem.db, 'a')).toThrow(/generation failure/)
+    expect(repos.listSetsForAgent(mem.db, 'a')).toEqual([])
+    expect(repos.getAgent(mem.db, 'a')!.policyGeneration).toBe(1)
+    mem.sqlite.exec('DROP TRIGGER reject_generation')
+    expect(repos.ensureSharedSetAssigned(mem.db, 'a')).toBe(true)
+    expect(repos.ensureSharedSetAssigned(mem.db, 'a')).toBe(false)
+    expect(repos.getAgent(mem.db, 'a')!.policyGeneration).toBe(2)
+  })
+
+  it('keeps warm and cold previews on the old snapshot while another DNS lookup is pending, then publishes once atomically', async () => {
+    const mem = hostnameFixture()
+    const before = evaluateAgentPolicy(mem.db, 'a')
+    let release!: (value: string[]) => void
+    const second = new Promise<string[]>((resolve) => { release = resolve })
+    dns.resolve4.mockImplementation((host) => host === 'second.test' ? second : Promise.resolve(['203.0.113.1']))
+    const refresh = refreshAllHostnameRules(mem.db)
+    await vi.waitFor(() => expect(dns.resolve4).toHaveBeenCalledWith('second.test'))
+    expect(repos.listResolvedForRule(mem.db, 'h1').map((row) => row.cidr)).toEqual(['192.0.2.1/32'])
+    expect(evaluateAgentPolicy(mem.db, 'a')).toBe(before)
+    policyCacheFor(mem.db).get('a', 'force-cold-preview', Date.now())
+    expect(evaluateAgentPolicy(mem.db, 'a').hash).toBe(before.hash)
+    release(['203.0.113.2']); await refresh
+    const after = evaluateAgentPolicy(mem.db, 'a')
+    expect(after.denyCidrs).toEqual(['203.0.113.1/32', '203.0.113.2/32'])
+    expect(after.generation).toBe(before.generation + 1)
+    await refreshAllHostnameRules(mem.db)
+    expect(evaluateAgentPolicy(mem.db, 'a')).toBe(after)
+  })
+
+  it('preserves failed lookup snapshots and never overwrites a newer publication made while DNS was pending', async () => {
+    const mem = hostnameFixture()
+    let reject!: (reason: Error) => void
+    const second = new Promise<string[]>((_, fail) => { reject = fail })
+    dns.resolve4.mockImplementation((host) => host === 'second.test' ? second : Promise.resolve(['203.0.113.1']))
+    const refresh = refreshAllHostnameRules(mem.db)
+    await vi.waitFor(() => expect(dns.resolve4).toHaveBeenCalledWith('second.test'))
+    repos.replaceResolvedForRule(mem.db, 'h1', ['198.51.100.1/32'])
+    repos.bumpAgentsForSet(mem.db, 'set-shared-default')
+    const newer = evaluateAgentPolicy(mem.db, 'a')
+    reject(Object.assign(new Error('timeout'), { code: 'ETIMEOUT' })); await refresh
+    expect(evaluateAgentPolicy(mem.db, 'a')).toBe(newer)
+    expect(newer.denyCidrs).toEqual(['192.0.2.2/32', '198.51.100.1/32'])
+  })
+})

apps/api/src/services/policy/cache-budget.test.ts

@@ -0,0 +1,198 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { POLICY_BUDGET, policyBudgetErrorEnvelopeSchema } from '@evofw/shared'
+import { evaluateAgentPolicy } from './evaluate.js'
+import { PolicyCache, policyCacheFor } from './cache.js'
+import { buildApp } from '../../app.js'
+import { loadConfig } from '../../config.js'
+import { hashToken } from '../../plugins/auth.js'
+import { createHash } from 'node:crypto'
+import { publishListCascade } from '../lists/entries.js'
+import { refreshAllHostnameRules } from './resolve-hostname.js'
+
+const dns = vi.hoisted(() => ({ resolve4: vi.fn(), resolve6: vi.fn(), cancel: vi.fn() }))
+vi.mock('node:dns/promises', async (original) => ({ ...await original<typeof import('node:dns/promises')>(), Resolver: class { resolve4 = dns.resolve4; resolve6 = dns.resolve6; cancel = dns.cancel } }))
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+const apps: Awaited<ReturnType<typeof buildApp>>[] = []
+afterEach(async () => { for (const mem of opened.splice(0)) mem.sqlite.close(); for (const app of apps.splice(0)) await app.close() })
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  repos.insertAgent(mem.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: 'a', status: 'approved' })
+  repos.ensureSharedSetAssigned(mem.db, 'a')
+  return mem
+}
+function source(mem: ReturnType<typeof memory>, n: number) {
+  repos.insertIpList(mem.db, { id: 'l', name: 'l', type: 'static' })
+  const insert = mem.sqlite.prepare("INSERT INTO ip_list_entries(id,list_id,cidr) VALUES(?,'l',?)")
+  mem.sqlite.transaction(() => { for (let i = 0; i < n; i++) insert.run(`c${i}`, `10.${i >>> 16}.${(i >>> 8) & 255}.${i & 255}/32`) })()
+}
+function rule(mem: ReturnType<typeof memory>, id = 'r', action: 'deny' | 'allow' = 'deny') {
+  repos.insertPolicyRule(mem.db, { id, setId: 'set-shared-default', priority: Number(id.slice(1)) || 10, action, listId: 'l' })
+}
+
+describe('bounded compiler and policy cache', () => {
+  it('accepts50k adjacent canonical memberships and rejects50001 including an overlapping covering CIDR before kernel collapse', () => {
+    const mem = memory(); source(mem, 50_000); rule(mem)
+    expect(evaluateAgentPolicy(mem.db, 'a').denyCidrs).toHaveLength(50_000)
+    mem.sqlite.prepare("INSERT INTO ip_list_entries(id,list_id,cidr) VALUES('extra','l','10.0.0.0/16')").run()
+    repos.bumpAgentsForList(mem.db, 'l')
+    expect(() => evaluateAgentPolicy(mem.db, 'a')).toThrow(/50000 effective/)
+    expect(repos.listIpListEntries(mem.db, 'l')).toHaveLength(50_001)
+  })
+
+  it('counts repeated raw list references before deduplication and rejects before unbounded expansion', () => {
+    const mem = memory(); source(mem, 50_000)
+    for (let i = 1; i <= 4; i++) rule(mem, `r${i}`)
+    expect(evaluateAgentPolicy(mem.db, 'a').denyCidrs).toHaveLength(50_000)
+    rule(mem, 'r5'); repos.bumpAgentsForSet(mem.db, 'set-shared-default')
+    expect(() => evaluateAgentPolicy(mem.db, 'a')).toThrow(/200000 raw/)
+  })
+
+  it('bounds source prefetch before materializing a huge set of unique list memberships', () => {
+    const mem = memory(); source(mem, 200_001); rule(mem)
+    expect(() => evaluateAgentPolicy(mem.db, 'a')).toThrow(/200000 raw/)
+  })
+
+  it('counts both transport expansions and actual source memberships, with all dualstack but explicit IPv4 scoped', () => {
+    const mem = memory()
+    repos.insertAgentPortRule(mem.db, { id: 'all', agentId: 'a', action: 'open', protocol: 'both', portStart: 22, portEnd: 22, srcKind: 'all' })
+    repos.insertAgentPortRule(mem.db, { id: 'v4', agentId: 'a', action: 'open', protocol: 'tcp', portStart: 23, portEnd: 23, srcKind: 'cidr', srcCidr: '0.0.0.0/0' })
+    const policy = evaluateAgentPolicy(mem.db, 'a')
+    expect(policy.portRules.filter((r) => r.id === 'all').map((r) => [r.id, r.protocol, r.srcCidrs])).toEqual([
+      ['all', 'tcp', ['0.0.0.0/0', '::/0']], ['all', 'udp', ['0.0.0.0/0', '::/0']],
+    ])
+    expect(policy.portRules.find((r) => r.id === 'v4')!.srcCidrs).toEqual(['0.0.0.0/0'])
+    const insert = mem.sqlite.prepare("INSERT INTO agent_port_rules(id,agent_id,action,protocol,port_start,port_end,src_kind) VALUES(?,'a','open','both',80,80,'all')")
+    mem.sqlite.transaction(() => { for (let i = 0; i < 500; i++) insert.run(`p${i}`) })()
+    repos.bumpAgentGeneration(mem.db, 'a')
+    expect(() => evaluateAgentPolicy(mem.db, 'a')).toThrow(/1000 serialized/)
+  })
+
+  it('counts shared L3 and expanded PortACL memberships together', () => {
+    const mem = memory(); source(mem, 25_000); rule(mem)
+    repos.insertAgentPortRule(mem.db, { id: 'p', agentId: 'a', action: 'open', protocol: 'both', portStart: 22, portEnd: 22, srcKind: 'list', listId: 'l' })
+    expect(() => evaluateAgentPolicy(mem.db, 'a')).toThrow(/50000 effective/)
+  })
+
+  it('rejects an oversized legacy JSON candidate before publishing rather than serving it truncated', () => {
+    const mem = memory()
+    repos.insertAgentPortRule(mem.db, { id: 'x'.repeat(POLICY_BUDGET.jsonBytes), agentId: 'a', action: 'close', protocol: 'tcp', portStart: 22, portEnd: 22, srcKind: 'all' })
+    expect(() => evaluateAgentPolicy(mem.db, 'a')).toThrow(/8MiB/)
+    expect(policyCacheFor(mem.db).stats.entries).toBe(0)
+  })
+
+  it('hits a DB-scoped cache, invalidates generations/settings/expiry, and interns identical content across agents', () => {
+    const mem = memory()
+    const now = '2026-01-01T00:00:00.000Z', expiry = '2026-01-01T00:01:00.000Z'
+    repos.insertOverride(mem.db, { id: 'o', agentId: 'a', action: 'deny', cidr: '8.8.8.8/32', expiresAt: expiry })
+    const first = evaluateAgentPolicy(mem.db, 'a', now)
+    expect(evaluateAgentPolicy(mem.db, 'a', now)).toBe(first)
+    repos.bumpAgentGeneration(mem.db, 'a')
+    const revised = evaluateAgentPolicy(mem.db, 'a', now)
+    expect(revised).not.toBe(first); expect(revised.hash).toBe(first.hash)
+    repos.setSetting(mem.db, 'agent_sync_interval_sec', '120')
+    expect(evaluateAgentPolicy(mem.db, 'a', now).syncIntervalSec).toBe(120)
+    expect(evaluateAgentPolicy(mem.db, 'a', expiry).denyCidrs).toEqual([])
+    repos.insertAgent(mem.db, { id: 'b', name: 'b', tokenPrefix: 'b', tokenHash: 'b', status: 'approved' })
+    const empty = evaluateAgentPolicy(mem.db, 'a', expiry), other = evaluateAgentPolicy(mem.db, 'b', expiry)
+    expect(empty.denyCidrs).toBe(other.denyCidrs)
+    expect(policyCacheFor(mem.db).stats).toMatchObject({ entries: 2, contents: 1 })
+    const secondDb = memory()
+    expect(evaluateAgentPolicy(secondDb.db, 'a', now)).not.toBe(empty)
+  })
+
+  it('evicts LRU entries and all unused intern references within conservative entry/byte budgets', () => {
+    const mem = memory(); const policy = evaluateAgentPolicy(mem.db, 'a')
+    const cache = new PolicyCache(2, 20_000)
+    cache.put('a', '1', policy, Infinity); cache.put('b', '1', policy, Infinity)
+    expect(cache.get('a', '1', 0)).toBeDefined()
+    cache.put('c', '1', { ...policy, hash: 'different', denyCidrs: ['8.8.8.8/32'] }, Infinity)
+    expect(cache.get('b', '1', 0)).toBeUndefined()
+    expect(cache.stats.entries).toBe(2)
+    cache.get('a', 'invalidated', 0); cache.get('c', 'invalidated', 0)
+    expect(cache.stats).toMatchObject({ entries: 0, contents: 0, bytes: 0 })
+    cache.put('huge', '1', { ...policy, chain: [{ ...policy.chain[0]!, sourceLabel: 'x'.repeat(20_000) }] }, Infinity)
+    expect(cache.stats.entries).toBe(0)
+    const bounded = new PolicyCache(128, 32 * 1024 * 1024)
+    for (let i = 0; i < 130; i++) bounded.put(String(i), '1', policy, Infinity)
+    expect(bounded.stats.entries).toBe(128)
+    expect(bounded.stats.contents).toBe(1)
+    expect(bounded.stats.bytes).toBeLessThanOrEqual(bounded.stats.maxBytes)
+    const pressure = new PolicyCache(128, 24_000)
+    for (let i = 0; i < 10; i++) pressure.put(String(i), '1', { ...policy, hash: String(i), denyCidrs: Array.from({ length: 50 }, (_, j) => `10.${i}.0.${j}/32`) }, Infinity)
+    expect(pressure.stats.entries).toBeGreaterThan(0)
+    expect(pressure.stats.entries).toBeLessThan(10)
+    expect(pressure.stats.contents).toBe(pressure.stats.entries)
+    expect(pressure.stats.bytes).toBeLessThanOrEqual(pressure.stats.maxBytes)
+  })
+
+  it('invalidates warm policies after actual PortACL list publication and hostname refresh, retaining unchanged cache hits', async () => {
+    const mem = memory()
+    repos.insertIpList(mem.db, { id: 'l', name: 'l', type: 'static' })
+    repos.insertAgentPortRule(mem.db, { id: 'p', agentId: 'a', action: 'open', protocol: 'tcp', portStart: 22, portEnd: 22, srcKind: 'list', listId: 'l' })
+    await publishListCascade(mem.db, 'l', { configJson: JSON.stringify({ items: [{ kind: 'ip', value: '8.8.8.8' }] }) })
+    const first = evaluateAgentPolicy(mem.db, 'a')
+    await publishListCascade(mem.db, 'l', { configJson: JSON.stringify({ items: [{ kind: 'ip', value: '9.9.9.9' }] }) })
+    const changed = evaluateAgentPolicy(mem.db, 'a')
+    expect(changed.hash).not.toBe(first.hash)
+    expect(changed.portRules[0]!.srcCidrs).toEqual(['9.9.9.9/32'])
+    await publishListCascade(mem.db, 'l')
+    expect(evaluateAgentPolicy(mem.db, 'a')).toBe(changed)
+    repos.insertPolicyRule(mem.db, { id: 'host', setId: 'set-shared-default', priority: 10, action: 'deny', hostname: 'test.example' })
+    dns.resolve4.mockResolvedValue(['192.0.2.1']); dns.resolve6.mockResolvedValue([])
+    await refreshAllHostnameRules(mem.db)
+    const resolved = evaluateAgentPolicy(mem.db, 'a')
+    expect(resolved.denyCidrs).toEqual(['192.0.2.1/32'])
+    dns.resolve4.mockResolvedValue(['192.0.2.2'])
+    await refreshAllHostnameRules(mem.db)
+    expect(evaluateAgentPolicy(mem.db, 'a').denyCidrs).toEqual(['192.0.2.2/32'])
+    expect(evaluateAgentPolicy(mem.db, 'a').hash).not.toBe(resolved.hash)
+  })
+
+  it('uses full revision-safe response ETags, preserves contact on304, rejects oversized candidates without changing lastgood, and exposes opt-in RouterOS budget errors', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } }); apps.push(app); await app.ready()
+    repos.insertAgent(app.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: hashToken('token'), status: 'approved' })
+    repos.ensureSharedSetAssigned(app.db, 'a')
+    const headers = { authorization: 'Bearer token' }
+    const first = await app.inject({ url: '/v1/agent/policy', headers })
+    expect(first.statusCode).toBe(200)
+    const etag = first.headers.etag!
+    expect(etag).toMatch(/^"policy-v1-[0-9a-f]{64}"$/)
+    expect(etag).toBe(`"policy-v1-${createHash('sha256').update(first.body).digest('hex')}"`)
+    repos.updateAgent(app.db, 'a', { lastSeenAt: '2000-01-01T00:00:00.000Z', appliedHash: first.json().hash, appliedRevision: first.json().generation })
+    const unchanged = await app.inject({ url: '/v1/agent/policy', headers: { ...headers, 'if-none-match': etag } })
+    expect(unchanged.statusCode).toBe(304); expect(unchanged.body).toBe('')
+    expect(repos.getAgent(app.db, 'a')!.lastSeenAt).not.toBe('2000-01-01T00:00:00.000Z')
+    repos.bumpAgentGeneration(app.db, 'a')
+    const revision = await app.inject({ url: '/v1/agent/policy', headers: { ...headers, 'if-none-match': etag } })
+    expect(revision.statusCode).toBe(200); expect(revision.headers.etag).not.toBe(etag)
+    expect(revision.json().hash).toBe(first.json().hash)
+    repos.setSetting(app.db, 'agent_sync_interval_sec', '120')
+    const setting = await app.inject({ url: '/v1/agent/policy', headers: { ...headers, 'if-none-match': revision.headers.etag! } })
+    expect(setting.statusCode).toBe(200); expect(setting.json().sync_interval_sec).toBe(120)
+    repos.insertOverride(app.db, { id: 'ttl', agentId: 'a', action: 'deny', cidr: '8.8.8.8/32', expiresAt: new Date(Date.now() + 60_000).toISOString() })
+    repos.bumpAgentGeneration(app.db, 'a')
+    const active = await app.inject({ url: '/v1/agent/policy', headers })
+    expect(active.json().hash).not.toBe(first.json().hash)
+    app.sqlite.prepare('UPDATE ip_overrides SET expires_at=? WHERE id=?').run(new Date(Date.now() - 1000).toISOString(), 'ttl')
+    const expired = await app.inject({ url: '/v1/agent/policy', headers: { ...headers, 'if-none-match': setting.headers.etag! } })
+    expect(expired.statusCode).toBe(200)
+    expect(expired.json().hash).toBe(first.json().hash)
+    expect(expired.json().generation).toBeGreaterThan(setting.json().generation)
+    expect(expired.headers.etag).not.toBe(setting.headers.etag)
+    const good = repos.getAgent(app.db, 'a')!
+    const insert = app.sqlite.prepare("INSERT INTO agent_port_rules(id,agent_id,action,protocol,port_start,port_end,src_kind) VALUES(?,'a','open','both',80,80,'all')")
+    app.sqlite.transaction(() => { for (let i = 0; i < 501; i++) insert.run(`p${i}`) })()
+    repos.bumpAgentGeneration(app.db, 'a')
+    const bad = await app.inject({ url: '/v1/agent/policy', headers })
+    expect(bad.statusCode).toBe(422); expect(bad.json().error.code).toBe('POLICY_BUDGET_EXCEEDED')
+    expect(repos.getAgent(app.db, 'a')).toMatchObject({ desiredHash: good.desiredHash, desiredHashGeneration: good.desiredHashGeneration, appliedHash: good.appliedHash, appliedRevision: good.appliedRevision })
+    const routeros = await app.inject({ url: '/v1/agent/policy?transport=routeros', headers })
+    expect(routeros.statusCode).toBe(200); expect(policyBudgetErrorEnvelopeSchema.safeParse(routeros.json()).success).toBe(true)
+    expect(routeros.json()).not.toHaveProperty('hash'); expect(routeros.headers.etag).toBeUndefined()
+    expect(routeros.headers['cache-control']).toBe('no-store')
+    expect((await app.inject({ url: '/v1/agent/policy?transport=anything', headers })).statusCode).toBe(400)
+    expect((await app.inject({ url: '/v1/agent/policy?transport=routeros' })).statusCode).toBe(401)
+  })
+})

apps/api/src/services/policy/cache.ts

@@ -0,0 +1,74 @@
+import type { Db } from '@evofw/db'
+import type { EvaluatedPolicy } from './evaluate.js'
+
+type Content = Pick<EvaluatedPolicy, 'denyCidrs' | 'allowCidrs' | 'portRules'>
+type Entry = { key: string; policy: EvaluatedPolicy; contentKey: string; bytes: number; expiresAt: number }
+type Intern = { content: Content; bytes: number; references: number }
+
+/** Conservative retained-heap accounting: UTF16 strings + object/array slots/headers.
+ * Repeated references are deliberately charged repeatedly within each representation.
+ * No serialized response bodies are retained. This is a bound on charged memory,
+ * not a measurement of a particular JavaScript engine's allocator or temporary work.
+ */
+export function retainedBytes(value: unknown): number {
+  if (value === null || value === undefined) return 16
+  if (typeof value === 'string') return 64 + value.length * 4
+  if (typeof value !== 'object') return 16
+  if (Array.isArray(value)) return 128 + value.reduce((sum, item) => sum + 16 + retainedBytes(item), 0)
+  return 128 + Object.entries(value).reduce((sum, [key, item]) => sum + 32 + retainedBytes(key) + retainedBytes(item), 0)
+}
+function freeze<T>(value: T): T {
+  if (value && typeof value === 'object' && !Object.isFrozen(value)) {
+    for (const child of Object.values(value)) freeze(child)
+    Object.freeze(value)
+  }
+  return value
+}
+
+export class PolicyCache {
+  private entries = new Map<string, Entry>()
+  private intern = new Map<string, Intern>()
+  private charged = 0
+  constructor(private maxEntries = 128, private maxBytes = 32 * 1024 * 1024) {}
+  get(agentId: string, key: string, now: number): EvaluatedPolicy | undefined {
+    const entry = this.entries.get(agentId)
+    if (!entry) return undefined
+    if (entry.key !== key || now >= entry.expiresAt) { this.remove(agentId); return undefined }
+    this.entries.delete(agentId); this.entries.set(agentId, entry)
+    return entry.policy
+  }
+  private remove(id: string) {
+    const entry = this.entries.get(id)
+    if (!entry) return
+    this.entries.delete(id); this.charged -= entry.bytes
+    const content = this.intern.get(entry.contentKey)!
+    if (--content.references === 0) { this.charged -= content.bytes; this.intern.delete(entry.contentKey) }
+  }
+  put(agentId: string, key: string, policy: EvaluatedPolicy, expiresAt: number): EvaluatedPolicy {
+    this.remove(agentId)
+    const { denyCidrs, allowCidrs, portRules, ...metadata } = policy
+    const bytes = retainedBytes({ agentId, key, metadata, expiresAt }) + 512
+    const content = { denyCidrs, allowCidrs, portRules }
+    const contentBytes = retainedBytes(content) + retainedBytes(policy.hash) + 512
+    const existing = this.intern.get(policy.hash)
+    if (bytes + contentBytes > this.maxBytes || this.maxEntries < 1) return policy
+    while (this.entries.size >= this.maxEntries || this.charged + bytes + (this.intern.has(policy.hash) ? 0 : contentBytes) > this.maxBytes) this.remove(this.entries.keys().next().value!)
+    // Eviction may have released the matching intern entry.
+    const intern = this.intern.get(policy.hash) ?? { content: freeze(existing?.content ?? content), bytes: contentBytes, references: 0 }
+    if (!this.intern.has(policy.hash)) { this.intern.set(policy.hash, intern); this.charged += contentBytes }
+    intern.references++
+    const stored = freeze({ ...metadata, ...intern.content })
+    this.entries.set(agentId, { key, policy: stored, contentKey: policy.hash, bytes, expiresAt })
+    this.charged += bytes
+    return stored
+  }
+  get stats() { return { entries: this.entries.size, contents: this.intern.size, bytes: this.charged, maxEntries: this.maxEntries, maxBytes: this.maxBytes } }
+}
+
+// Closing a DB and dropping its last reference releases all cache content.
+const caches = new WeakMap<Db, PolicyCache>()
+export function policyCacheFor(db: Db) {
+  let cache = caches.get(db)
+  if (!cache) { cache = new PolicyCache(); caches.set(db, cache) }
+  return cache
+}

apps/api/src/services/policy/evaluate.ts

@@ -5,8 +5,17 @@ import {
   defaultActionFromLegacyMode,
   legacyModeFromDefaultAction,
   type DefaultAction,
+  cidrContains,
+  type ExplainMatch,
+  POLICY_BUDGET, POLICY_BUDGET_ERROR,
 } from '@evofw/shared'
 import { uniqCidrs } from '../uniq.js'
+import { AppError } from '../../plugins/error-handler.js'
+import { policyCacheFor } from './cache.js'
+
+export function policyBudgetExceeded(reason: string): never {
+  throw new AppError(POLICY_BUDGET_ERROR, `Policy budget exceeded: ${reason}`, 422)
+}
 
 export const POLICY_APPLY_VERSION = 3 as const
 
@@ -80,9 +89,18 @@ function buildExpansionContext(
   const hostnameRuleIds = [
     ...new Set(rules.filter((r) => r.hostname?.trim()).map((r) => r.id)),
   ]
+  let entriesByList: Map<string, string[]>, resolvedByRule: Map<string, string[]>
+  try {
+    entriesByList = repos.mapIpListEntriesByListIds(db, listIds, POLICY_BUDGET.rawMemberships)
+    const loaded = [...entriesByList.values()].reduce((n, cidrs) => n + cidrs.length, 0)
+    resolvedByRule = repos.mapResolvedCidrsByRuleIds(db, hostnameRuleIds, POLICY_BUDGET.rawMemberships - loaded)
+  } catch (error) {
+    if (error instanceof RangeError) policyBudgetExceeded('200000 raw memberships')
+    throw error
+  }
   return {
-    entriesByList: repos.mapIpListEntriesByListIds(db, listIds),
+    entriesByList,
-    resolvedByRule: repos.mapResolvedCidrsByRuleIds(db, hostnameRuleIds),
+    resolvedByRule,
     listNames: repos.mapIpListNames(db, listIds),
   }
 }
@@ -138,7 +156,7 @@ function expandPortSrcCidrs(
     listId: string | null
   },
 ): string[] {
-  if (row.srcKind === 'all') return ['0.0.0.0/0']
+  if (row.srcKind === 'all') return ['0.0.0.0/0', '::/0']
   if (row.srcKind === 'cidr' && row.srcCidr?.trim()) {
     return [row.srcCidr.trim()]
   }
@@ -154,6 +172,7 @@ function expandPortSrcCidrs(
 function expandPortRules(
   ctx: ExpansionContext,
   rows: ReturnType<typeof repos.listEnabledAgentPortRules>,
+  budget: { raw: number; effective: number },
 ): EvaluatedPortRule[] {
   const out: EvaluatedPortRule[] = []
   for (const row of rows) {
@@ -169,6 +188,11 @@ function expandPortRules(
           ? ['tcp', 'udp']
           : ['tcp']
     for (const protocol of protocols) {
+      budget.raw += row.srcKind === 'list' ? (ctx.entriesByList.get(row.listId?.trim() ?? '')?.length ?? 0) : srcCidrs.length
+      budget.effective += srcCidrs.length
+      if (budget.raw > POLICY_BUDGET.rawMemberships) policyBudgetExceeded('200000 raw memberships')
+      if (budget.effective > POLICY_BUDGET.effectiveMemberships) policyBudgetExceeded('50000 effective memberships')
+      if (out.length >= POLICY_BUDGET.serializedRules) policyBudgetExceeded('1000 serialized port rules')
       out.push({
         id: row.id,
         action,
@@ -183,7 +207,33 @@ function expandPortRules(
 }
 
 /** Evaluate allow/deny sets for an agent from assigned policy sets. */
-export function evaluateAgentPolicy(db: Db, agentId: string): EvaluatedPolicy {
+export function evaluateAgentPolicy(db: Db, agentId: string, now = new Date().toISOString()): EvaluatedPolicy {
+  const pin = repos.getPolicyPin(db, agentId)
+  if (pin) {
+    const agent = repos.getAgent(db, agentId)!
+    const syncIntervalSec = Number(repos.getSetting(db, 'agent_sync_interval_sec') || '60') || 60
+    const cache = policyCacheFor(db)
+    const key = JSON.stringify(['pin', pin.snapshot_id, agent.policyGeneration, syncIntervalSec])
+    const hit = cache.get(agentId, key, Date.parse(now))
+    if (hit) return hit
+    const snapshot = repos.getPolicySnapshot(db, pin.snapshot_id)!
+    return cache.put(agentId, key, { ...JSON.parse(snapshot.payload_json), generation: agent.policyGeneration, syncIntervalSec }, Infinity)
+  }
+  // Expiry publication/bump must run before even consulting a cached revision.
+  repos.expireOverridesForAgent(db, agentId, now)
+  const agent = repos.getAgent(db, agentId)
+  if (!agent) throw new Error(`agent not found: ${agentId}`)
+  const key = JSON.stringify([agent.policyGeneration, agent.defaultAction, repos.getSetting(db, 'agent_sync_interval_sec')])
+  const cache = policyCacheFor(db)
+  const hit = cache.get(agentId, key, Date.parse(now))
+  if (hit) return hit
+  const policy = compileAgentPolicy(db, agentId, now)
+  const expires = repos.nextOverrideExpiry(db, agentId)
+  return cache.put(agentId, key, policy, expires ? Date.parse(expires) : Infinity)
+}
+
+export function compileAgentPolicy(db: Db, agentId: string, now: string): EvaluatedPolicy {
+  repos.expireOverridesForAgent(db, agentId, now)
   const agent = repos.getAgent(db, agentId)
   if (!agent) {
     throw new Error(`agent not found: ${agentId}`)
@@ -193,12 +243,20 @@ export function evaluateAgentPolicy(db: Db, agentId: string): EvaluatedPolicy {
     .listSetsForAgent(db, agentId)
     .filter((s) => s.enabled === 1)
   const ordered = repos.listPolicyRulesForAgent(db, agentId)
-  const overrides = repos.listOverrides(db, agentId)
+  const overrides = repos.listActiveOverrides(db, agentId, now)
   const portRuleRows = repos.listEnabledAgentPortRules(db, agentId)
+  if (portRuleRows.reduce((n, row) => n + (row.protocol === 'both' ? 2 : 1), 0) > POLICY_BUDGET.serializedRules) policyBudgetExceeded('1000 serialized port rules')
+  if (ordered.length + overrides.length > POLICY_BUDGET.rawMemberships) policyBudgetExceeded('policy metadata cardinality')
   const ctx = buildExpansionContext(db, ordered, portRuleRows)
 
-  const deny: string[] = []
+  const deny = new Set<string>()
-  const allow: string[] = []
+  const allow = new Set<string>()
+  const budget = { raw: 0, effective: 0 }
+  const addSource = (target: Set<string>, cidrs: string[]) => {
+    budget.raw += cidrs.length
+    if (budget.raw > POLICY_BUDGET.rawMemberships) policyBudgetExceeded('200000 raw memberships')
+    for (const cidr of cidrs) target.add(cidr)
+  }
   const chain: PolicyChainStep[] = []
   let rulesDeny = 0
   let rulesAllow = 0
@@ -207,10 +265,10 @@ export function evaluateAgentPolicy(db: Db, agentId: string): EvaluatedPolicy {
     const cidrs = expandRule(ctx, rule)
     const action = rule.action === 'deny' ? 'deny' : 'allow'
     if (action === 'deny') {
-      deny.push(...cidrs)
+      addSource(deny, cidrs)
       rulesDeny += 1
     } else {
-      allow.push(...cidrs)
+      addSource(allow, cidrs)
       rulesAllow += 1
     }
     const src = sourceMeta(ctx, rule)
@@ -229,8 +287,7 @@ export function evaluateAgentPolicy(db: Db, agentId: string): EvaluatedPolicy {
 
   for (const o of overrides) {
     const action = o.action === 'deny' ? 'deny' : 'allow'
-    if (action === 'deny') deny.push(o.cidr)
+    addSource(action === 'deny' ? deny : allow, [o.cidr])
-    else allow.push(o.cidr)
     chain.push({
       setId: null,
       setName: null,
@@ -242,24 +299,26 @@ export function evaluateAgentPolicy(db: Db, agentId: string): EvaluatedPolicy {
     })
   }
 
-  const denyCidrs = uniqCidrs(deny)
+  const denyCidrs = uniqCidrs([...deny])
   const denySet = new Set(denyCidrs)
-  const allowRaw = uniqCidrs(allow)
+  const allowRaw = uniqCidrs([...allow])
   const allowCidrs = allowRaw.filter((c) => !denySet.has(c))
   const conflictsDropped = allowRaw.length - allowCidrs.length
   const defaultAction = resolveDefaultAction(agent.defaultAction)
   const policyMode = legacyModeFromDefaultAction(defaultAction)
-  const portRules = expandPortRules(ctx, portRuleRows)
+  budget.effective = denyCidrs.length + allowCidrs.length
+  if (budget.effective > POLICY_BUDGET.effectiveMemberships) policyBudgetExceeded('50000 effective memberships')
+  const portRules = expandPortRules(ctx, portRuleRows, budget)
 
   const payload = JSON.stringify({
     apply_version: POLICY_APPLY_VERSION,
-    generation: agent.policyGeneration,
     defaultAction,
     denyCidrs,
     allowCidrs,
     portRules,
   })
   const hash = `sha256:${createHash('sha256').update(payload).digest('hex')}`
+  if (Buffer.byteLength(payload) > POLICY_BUDGET.jsonBytes) policyBudgetExceeded('8MiB policy JSON')
 
   const syncIntervalSec =
     Number(repos.getSetting(db, 'agent_sync_interval_sec') || '60') || 60
@@ -293,3 +352,44 @@ export function truncateCidrs(cidrs: string[], limit: number): string[] {
   if (limit <= 0) return []
   return cidrs.slice(0, limit)
 }
+
+/** Bounded source witnesses use the compiler's own expansion and active overrides. */
+export function policySourceWitnesses(db: Db, agentId: string, ip: string, now = new Date().toISOString()) {
+  if (repos.getPolicyPin(db, agentId)) {
+    const policy = evaluateAgentPolicy(db, agentId, now)
+    const matches: ExplainMatch[] = []
+    for (const [stage, cidrs] of [['deny', policy.denyCidrs], ['allow', policy.allowCidrs]] as const) {
+      const cidr = cidrs.find((network) => cidrContains(ip, network))
+      if (cidr) matches.push({ rule_id: null, set_id: null, list_id: null, override_id: null, cidr,
+        label: 'Frozen policy snapshot; original source attribution unavailable', stage })
+    }
+    return { matches, total: matches.length, deny: matches.find((match) => match.stage === 'deny') ?? null, allow: matches.find((match) => match.stage === 'allow') ?? null }
+  }
+  const rules = repos.listPolicyRulesForAgent(db, agentId)
+  const ctx = buildExpansionContext(db, rules, [])
+  const sets = new Map(repos.listSetsForAgent(db, agentId).map((set) => [set.setId, set.name]))
+  const matches: ExplainMatch[] = []
+  let total = 0
+  let deny: ExplainMatch | null = null, allow: ExplainMatch | null = null
+  const matchCache = new Map<string, string | null>()
+  const add = (match: ExplainMatch) => {
+    total++
+    if (matches.length < 100) matches.push(match)
+    if (match.stage === 'deny' && !deny) deny = match
+    if (match.stage === 'allow' && !allow) allow = match
+  }
+  for (const rule of rules) {
+    const key = rule.listId ? `list:${rule.listId}` : rule.hostname ? `host:${rule.id}` : `cidr:${rule.cidr}`
+    if (!matchCache.has(key)) matchCache.set(key, expandRule(ctx, rule).find((network) => cidrContains(ip, network)) ?? null)
+    const cidr = matchCache.get(key)
+    if (!cidr) continue
+    const source = sourceMeta(ctx, rule)
+    add({ rule_id: rule.id, set_id: rule.setId, list_id: rule.listId, override_id: null, cidr,
+      label: `${sets.get(rule.setId) ?? rule.setId}: ${source.label}`, stage: rule.action === 'deny' ? 'deny' : 'allow' })
+  }
+  for (const override of repos.listActiveOverrides(db, agentId, now)) if (cidrContains(ip, override.cidr)) {
+    add({ rule_id: null, set_id: null, list_id: null, override_id: override.id, cidr: override.cidr,
+      label: override.reason || override.comment || override.cidr, stage: override.action === 'deny' ? 'deny' : 'allow' })
+  }
+  return { matches, total, deny, allow }
+}

apps/api/src/services/policy/explain.test.ts

@@ -0,0 +1,166 @@
+import { afterEach, describe, expect, it } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { explainQuerySchema, explainResponseSchema } from '@evofw/shared'
+import { explainAgentPolicy } from './explain.js'
+import { evaluateAgentPolicy } from './evaluate.js'
+import { readFileSync } from 'node:fs'
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+afterEach(() => { for (const mem of opened.splice(0)) mem.sqlite.close() })
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  repos.insertAgent(mem.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: 'a', status: 'approved' })
+  repos.ensureSharedSetAssigned(mem.db, 'a')
+  return mem
+}
+const now = '2026-01-01T00:00:00.000Z'
+const later = '2026-01-01T00:02:00.000Z'
+
+describe('policy explain and override expiry', () => {
+  it('follows the actual iptables compiler INPUT-only chain without nft conntrack bypass', () => {
+    const script = readFileSync(new URL('../../agent-scripts/evofw-firewall.sh', import.meta.url), 'utf8')
+    const compiler = script.slice(script.indexOf('candidate = ["*filter",'), script.indexOf('rollback = ["*filter",'))
+    expect(compiler).toContain('--match-set evofw_deny_v{family} src -j DROP')
+    expect(compiler).toContain('--match-set evofw_allow_v{family} src -j ACCEPT')
+    expect(compiler).toContain('-I INPUT 1 -j EVOFW_INPUT')
+    expect(compiler).not.toMatch(/FORWARD|ESTABLISHED|RELATED/)
+    const { db } = memory()
+    repos.updateAgent(db, 'a', { lastApplyKernelMethod: 'ipset' })
+    repos.insertPolicyRule(db, { id: 'deny', setId: 'set-shared-default', priority: 10, action: 'deny', cidr: '10.0.0.0/8' })
+    repos.insertPolicyRule(db, { id: 'allow', setId: 'set-shared-default', priority: 20, action: 'allow', cidr: '192.0.2.0/24' })
+    for (const state of ['new', 'established', 'related']) {
+      expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '10.0.0.1', connection_state: state }), now).desired).toMatchObject({ decision: 'drop', stage: 'deny' })
+      expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '10.0.0.1', chain: 'forward', connection_state: state }), now).desired).toMatchObject({ decision: null, stage: 'external' })
+    }
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '192.0.2.1' }), now).desired).toMatchObject({ decision: 'accept', stage: 'allow' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8' }), now).desired).toMatchObject({ decision: null, stage: 'external' })
+    repos.updateAgent(db, 'a', { defaultAction: 'drop' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8' }), now).desired).toMatchObject({ decision: 'drop', stage: 'default' })
+  })
+  it('bounds active overrides and leaves target assignments/defaults/overrides intact when clone exceeds the limit', () => {
+    const { db, sqlite } = memory()
+    repos.insertAgent(db, { id: 'b', name: 'b', tokenPrefix: 'b', tokenHash: 'b', status: 'approved', defaultAction: 'drop' })
+    repos.insertPolicySet(db, { id: 'keep', name: 'Keep', enabled: 1 })
+    repos.setAgentPolicySets(db, 'b', ['keep'])
+    repos.insertOverride(db, { id: 'target', agentId: 'b', cidr: '8.8.8.8/32', action: 'allow' })
+    const statement = sqlite.prepare("INSERT INTO ip_overrides(id,agent_id,cidr,action) VALUES(?,'a',?,'deny')")
+    sqlite.transaction(() => { for (let i = 0; i < 1000; i++) statement.run(`o-${i}`, `10.${i >> 8}.${i & 255}.1/32`) })()
+    expect(() => repos.insertOverride(db, { id: 'overflow', agentId: 'a', cidr: '9.9.9.9/32', action: 'deny' })).toThrow(/1000/)
+    statement.run('legacy-over-limit', '9.9.9.9/32')
+    const beforeAgent = repos.getAgent(db, 'b'), beforeSets = repos.listSetsForAgent(db, 'b'), beforeOverrides = repos.listOverrides(db, 'b')
+    expect(() => repos.cloneRulesFrom(db, 'a', 'b', true)).toThrow(/1000/)
+    expect(repos.getAgent(db, 'b')).toEqual(beforeAgent)
+    expect(repos.listSetsForAgent(db, 'b')).toEqual(beforeSets)
+    expect(repos.listOverrides(db, 'b')).toEqual(beforeOverrides)
+  })
+
+  it('keeps at most100 recent expired overrides/agent and7days after one expiry generation bump', () => {
+    const { db, sqlite } = memory()
+    const statement = sqlite.prepare("INSERT INTO ip_overrides(id,agent_id,cidr,action,expires_at,reason) VALUES(?,'a',?,'deny',?,'Archived reason')")
+    sqlite.transaction(() => {
+      for (let i = 0; i < 150; i++) statement.run(`old-${i}`, `10.0.${i}.1/32`, '2026-01-01T00:01:00.000Z')
+      statement.run('too-old', '9.9.9.9/32', '2025-12-20T00:00:00.000Z')
+    })()
+    const generation = repos.getAgent(db, 'a')!.policyGeneration
+    expect(repos.expireOverridesForAgent(db, 'a', later)).toBe(151)
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(generation + 1)
+    expect(repos.listOverrides(db, 'a')).toHaveLength(100)
+    expect(repos.listOverrides(db, 'a').some((row) => row.id === 'too-old')).toBe(false)
+    expect(repos.listOverrides(db, 'a')[0]!.reason).toBe('Archived reason')
+    expect(repos.expireOverridesForAgent(db, 'a', later)).toBe(0)
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(generation + 1)
+  })
+
+  it('follows emitted nft precedence deny > close > open > implicit close > allow > default', () => {
+    const { db } = memory()
+    repos.insertPolicyRule(db, { id: 'deny', setId: 'set-shared-default', priority: 10, action: 'deny', cidr: '10.0.0.0/8' })
+    repos.insertPolicyRule(db, { id: 'allow', setId: 'set-shared-default', priority: 20, action: 'allow', cidr: '0.0.0.0/0' })
+    for (const [id, action, srcCidr, priority] of [['open', 'open', '192.0.2.0/24', 1], ['close', 'close', '192.0.2.1/32', 999]] as const) repos.insertAgentPortRule(db, { id, agentId: 'a', action, protocol: 'tcp', portStart: 22, portEnd: 22, srcKind: 'cidr', srcCidr, priority })
+    const cases = [
+      ['10.0.0.1', 22, 'drop', 'deny'], ['192.0.2.1', 22, 'drop', 'port_close'],
+      ['192.0.2.2', 22, 'accept', 'port_open'], ['203.0.113.1', 22, 'drop', 'port_implicit_close'],
+      ['203.0.113.1', 443, 'accept', 'allow'], ['2001:db8::1', 22, 'drop', 'port_implicit_close'],
+      ['2001:db8::1', 443, 'accept', 'default'],
+    ] as const
+    for (const [ip, port, decision, stage] of cases) {
+      const result = explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip, port, protocol: 'tcp' }), now)
+      expect(result.desired).toMatchObject({ decision, stage })
+      expect(explainResponseSchema.safeParse(result).success).toBe(true)
+      expect(result.limits).toContain('external_firewall_may_override_accept')
+      expect(result.applied.same_as_desired).toBeNull()
+    }
+    const withoutTransport = explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '203.0.113.1' }), now)
+    expect(withoutTransport.desired).toMatchObject({ decision: null, stage: 'transport_unknown' })
+    const established = explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '10.0.0.1', protocol: 'tcp', port: 22, connection_state: 'established' }), now)
+    expect(established.desired).toMatchObject({ decision: 'accept', stage: 'stateful' })
+  })
+
+  it('scopes forward port ACL to DNAT and distinguishes RouterOS external and unsupported cases', () => {
+    const { db } = memory()
+    repos.insertAgentPortRule(db, { id: 'close', agentId: 'a', action: 'close', protocol: 'tcp', portStart: 22, portEnd: 22, srcKind: 'all' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8', port: 22, protocol: 'tcp', chain: 'forward' }), now).desired).toMatchObject({ decision: 'accept', stage: 'default' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8', port: 22, protocol: 'tcp', chain: 'forward', dnat: 'true' }), now).desired).toMatchObject({ decision: 'drop', stage: 'port_close' })
+    repos.updateAgent(db, 'a', { platform: 'mikrotik', defaultAction: 'drop' })
+    repos.deleteAgentPortRule(db, 'close')
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8' }), now).desired).toMatchObject({ decision: null, stage: 'external' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8', chain: 'forward' }), now).desired).toMatchObject({ decision: 'drop', stage: 'default' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '2001:db8::1', chain: 'forward' }), now).desired).toMatchObject({ decision: null, stage: 'unsupported' })
+    repos.updateAgent(db, 'a', { telemetryVersion: 2, capabilitiesJson: JSON.stringify(['policy-ipv6']), clientVersion: 'rsc/9' })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '2001:db8::1', chain: 'forward' }), now).desired).toMatchObject({ decision: 'drop', stage: 'default' })
+  })
+
+  it('marks the entire desired candidate unsupported when reported runtime cannot apply its ACL/IPv6 features', () => {
+    const { db } = memory()
+    repos.insertPolicyRule(db, { id: 'deny', setId: 'set-shared-default', priority: 10, action: 'deny', cidr: '10.0.0.0/8' })
+    repos.insertAgentPortRule(db, { id: 'open', agentId: 'a', action: 'open', protocol: 'tcp', portStart: 22, portEnd: 22, srcKind: 'all' })
+    repos.updateAgent(db, 'a', { lastApplyKernelMethod: 'ipset', telemetryVersion: 2, capabilitiesJson: JSON.stringify(['policy_ipv4']) })
+    const rejected = explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '10.0.0.1', port: 22, protocol: 'tcp' }), now)
+    expect(rejected.desired).toMatchObject({ decision: null, stage: 'unsupported' })
+    expect(rejected.limits).toContain('port_acl_requires_nft_backend')
+    repos.updateAgent(db, 'a', { lastApplyKernelMethod: 'nft' })
+    repos.insertPolicyRule(db, { id: 'v6', setId: 'set-shared-default', priority: 20, action: 'deny', cidr: '2001:db8::/32' })
+    repos.bumpAgentGeneration(db, 'a')
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '10.0.0.1' }), now).desired).toMatchObject({ decision: null, stage: 'unsupported' })
+    repos.updateAgent(db, 'a', { capabilitiesJson: JSON.stringify(['policy-ipv6', 'policy-port-acl-ipv6']) })
+    expect(explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '2001:db8::1' }), now).desired).toMatchObject({ decision: 'drop', stage: 'deny' })
+  })
+
+  it('uses compiler list/hostname/override witnesses while preserving deny-wins semantics', () => {
+    const { db } = memory()
+    repos.insertIpList(db, { id: 'l', name: 'Source list', type: 'static' }); repos.replaceIpListEntries(db, 'l', ['192.0.2.0/24'])
+    repos.insertPolicyRule(db, { id: 'r', setId: 'set-shared-default', priority: 10, action: 'deny', listId: 'l' })
+    repos.insertOverride(db, { id: 'o', agentId: 'a', cidr: '192.0.2.1/32', action: 'allow', reason: 'Temporary diagnosis', expiresAt: later, createdByUserId: 'operator' })
+    const result = explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '192.0.2.1' }), now)
+    expect(result.desired.match).toMatchObject({ rule_id: 'r', list_id: 'l', cidr: '192.0.2.0/24' })
+    expect(result.matches.some((match) => match.override_id === 'o')).toBe(true)
+    expect(result.desired.decision).toBe('drop')
+    expect(result.limits).toContain('source_attribution_is_logical_witness')
+  })
+
+  it('expires overrides before computing hash, increments once, preserves author/reason and never resurrects on a backward clock', () => {
+    const { db } = memory()
+    repos.insertOverride(db, { id: 'o', agentId: 'a', cidr: '8.8.8.8/32', action: 'deny', reason: 'Investigation', createdByUserId: 'operator', createdByName: 'Operator', expiresAt: later })
+    const before = evaluateAgentPolicy(db, 'a', now)
+    expect(before.denyCidrs).toEqual(['8.8.8.8/32'])
+    const expired = evaluateAgentPolicy(db, 'a', later)
+    expect(expired.denyCidrs).toEqual([])
+    expect(expired.hash).not.toBe(before.hash)
+    expect(expired.generation).toBe(before.generation + 1)
+    expect(evaluateAgentPolicy(db, 'a', later).generation).toBe(expired.generation)
+    expect(evaluateAgentPolicy(db, 'a', now).denyCidrs).toEqual([])
+    expect(repos.listOverrides(db, 'a')[0]).toMatchObject({ reason: 'Investigation', createdByUserId: 'operator', expiredAt: later })
+  })
+
+  it('bounds witnesses and reports actual desired/applied differences without claiming external rules are complete', () => {
+    const { db } = memory()
+    for (let i = 0; i < 110; i++) repos.insertPolicyRule(db, { id: `r-${i}`, setId: 'set-shared-default', priority: i + 1, action: 'allow', cidr: '0.0.0.0/0' })
+    const desired = evaluateAgentPolicy(db, 'a', now)
+    repos.updateAgent(db, 'a', { appliedHash: desired.hash, appliedRevision: desired.generation })
+    const result = explainAgentPolicy(db, 'a', explainQuerySchema.parse({ ip: '8.8.8.8' }), now)
+    expect(result.matches).toHaveLength(100)
+    expect(result.matches_total).toBe(110)
+    expect(result.truncated).toBe(true)
+    expect(result.applied.same_as_desired).toBe(true)
+    expect(result.external_firewall.complete).toBe(false)
+  })
+})

apps/api/src/services/policy/explain.ts

@@ -0,0 +1,92 @@
+import { repos, type Db } from '@evofw/db'
+import { cidrContains, type ExplainMatch, type ExplainQuery, type ExplainResponse } from '@evofw/shared'
+import { evaluateAgentPolicy, policySourceWitnesses } from './evaluate.js'
+import { mapAgentDiagnostics } from '../row-mappers.js'
+
+/** Explain desired new-flow rules; never equate a configured accept with observed delivery. */
+export function explainAgentPolicy(db: Db, agentId: string, query: ExplainQuery, now = new Date().toISOString()): ExplainResponse {
+  const policy = evaluateAgentPolicy(db, agentId, now)
+  const agent = repos.getAgent(db, agentId)!
+  const sources = policySourceWitnesses(db, agentId, query.ip, now)
+  const matches = [...sources.matches]
+  let matchesTotal = sources.total
+  const add = (match: ExplainMatch) => { matchesTotal++; if (matches.length < 100) matches.push(match); return match }
+  const limits = ['desired_policy_not_kernel_observation', 'external_firewall_may_override_accept', 'non_loopback_ingress_assumed']
+  const diagnostics = mapAgentDiagnostics({ ...agent, desiredHash: policy.hash, desiredHashGeneration: policy.generation })
+  const needsIpv6 = policy.denyCidrs.some((cidr) => cidr.includes(':')) || policy.allowCidrs.some((cidr) => cidr.includes(':')) || policy.portRules.some((rule) => rule.srcCidrs.some((cidr) => cidr.includes(':')))
+  const knownCapabilities = agent.telemetryVersion !== null
+  const ipsetBackend = agent.platform === 'linux' && /ipset|iptables/.test(agent.lastApplyKernelMethod ?? '')
+  const ipsetAclUnsupported = ipsetBackend && policy.portRules.length > 0
+  const mikrotikAclUnsupported = agent.platform === 'mikrotik' && policy.portRules.length > 0
+  const ipv6Unsupported = (needsIpv6 || query.ip.includes(':')) && (knownCapabilities ? !diagnostics.capabilities.includes('policy-ipv6') : agent.platform === 'mikrotik')
+  const ipv6AclUnsupported = agent.platform === 'linux' && knownCapabilities && policy.portRules.some((rule) => rule.srcCidrs.some((cidr) => cidr.includes(':'))) && !diagnostics.capabilities.includes('policy-port-acl-ipv6')
+  let decision: 'accept' | 'drop' | null = null
+  let stage: ExplainResponse['desired']['stage'] = 'default'
+  let match: ExplainMatch | null = null
+  const deny = policy.denyCidrs.find((network) => cidrContains(query.ip, network))
+  const allow = policy.allowCidrs.find((network) => cidrContains(query.ip, network))
+  const portApplicable = agent.platform === 'linux' && query.port !== undefined && (query.chain === 'input' || query.dnat)
+  const portCandidates = portApplicable ? policy.portRules.filter((rule) => rule.protocol === query.protocol && query.port! >= rule.portStart && query.port! <= rule.portEnd) : []
+  const portMatches = new Map<string, ExplainMatch>()
+  for (const rule of portCandidates) {
+    const cidr = rule.srcCidrs.find((network) => cidrContains(query.ip, network))
+    if (!cidr) continue
+    const item = add({ rule_id: rule.id, set_id: null, list_id: null, override_id: null, cidr,
+      label: `${rule.protocol}/${rule.portStart === rule.portEnd ? rule.portStart : `${rule.portStart}-${rule.portEnd}`}`,
+      stage: rule.action === 'close' ? 'port_close' : 'port_open' })
+    portMatches.set(`${rule.id}:${rule.protocol}`, item)
+  }
+  const close = portCandidates.find((rule) => rule.action === 'close' && portMatches.has(`${rule.id}:${rule.protocol}`))
+  const open = portCandidates.find((rule) => rule.action === 'open' && portMatches.has(`${rule.id}:${rule.protocol}`))
+  const implicit = portCandidates.find((rule) => rule.action === 'open')
+  if (ipsetAclUnsupported || mikrotikAclUnsupported || ipv6Unsupported || ipv6AclUnsupported) {
+    stage = 'unsupported'
+    if (ipsetAclUnsupported) limits.push('port_acl_requires_nft_backend')
+    if (mikrotikAclUnsupported) limits.push('mikrotik_port_acl_not_supported')
+    if (ipv6Unsupported) limits.push('ipv6_policy_capability_not_reported')
+    if (ipv6AclUnsupported) limits.push('ipv6_port_acl_capability_not_reported')
+  } else if (ipsetBackend && query.chain === 'forward') {
+    stage = 'external'; limits.push('ipset_forward_chain_not_managed')
+  } else if (agent.platform === 'linux' && !ipsetBackend && query.connection_state !== 'new') {
+    stage = 'stateful'; decision = 'accept'; limits.push('established_related_precedes_owned_policy')
+  } else if (deny) {
+    stage = 'deny'; decision = 'drop'; match = sources.deny
+  } else if (close) {
+    stage = 'port_close'; decision = 'drop'; match = portMatches.get(`${close.id}:${close.protocol}`)!
+  } else if (open) {
+    stage = 'port_open'; decision = 'accept'; match = portMatches.get(`${open.id}:${open.protocol}`)!
+  } else if (implicit) {
+    stage = 'port_implicit_close'; decision = 'drop'
+    match = add({ rule_id: implicit.id, set_id: null, list_id: null, override_id: null, cidr: null,
+      label: `${implicit.protocol}/${implicit.portStart}-${implicit.portEnd}: unmatched source`, stage })
+  } else if (agent.platform === 'linux' && query.port === undefined && policy.portRules.length && (query.chain === 'input' || query.dnat)) {
+    stage = 'transport_unknown'; limits.push('port_protocol_required_for_l4_decision')
+  } else if (agent.platform === 'mikrotik' && query.chain === 'input') {
+    stage = 'external'; limits.push('mikrotik_input_allow_and_default_are_external')
+  } else if (allow) {
+    stage = 'allow'; decision = 'accept'; match = sources.allow
+  } else if ((agent.platform === 'mikrotik' || ipsetBackend) && policy.defaultAction === 'accept') {
+    stage = 'external'; limits.push(ipsetBackend ? 'ipset_accept_default_is_external_fallthrough' : 'mikrotik_accept_default_is_external_fallthrough')
+  } else {
+    stage = 'default'; decision = policy.defaultAction
+  }
+  if (query.chain === 'forward') limits.push(query.dnat ? 'forward_port_acl_dnat_scope' : 'forward_port_acl_skipped_without_dnat')
+  if ((needsIpv6 || query.ip.includes(':')) && !knownCapabilities && agent.platform === 'linux') limits.push('ipv6_runtime_capability_unknown')
+  if (match && (stage === 'deny' || stage === 'allow')) limits.push('source_attribution_is_logical_witness')
+  const snapshot = repos.getHostFirewallSnapshot(db, agentId)
+  let externalRules = 0
+  if (snapshot) {
+    try {
+      const payload = JSON.parse(snapshot.payloadJson) as { rules?: { ownership?: string }[] }
+      externalRules = payload.rules?.filter((rule) => rule.ownership === 'foreign').length ?? 0
+    } catch { /* unavailable snapshots never become proof of external absence */ }
+  }
+  return {
+    agent_id: agentId,
+    query: { ip: query.ip, port: query.port ?? null, protocol: query.protocol ?? null, chain: query.chain, dnat: query.dnat, connection_state: query.connection_state },
+    desired: { revision: policy.generation, hash: policy.hash, decision, stage, match },
+    applied: { revision: agent.appliedRevision, hash: agent.appliedHash, same_as_desired: agent.appliedHash ? agent.appliedHash === policy.hash && agent.appliedRevision === policy.generation : null, status: diagnostics.status },
+    matches, matches_total: matchesTotal, truncated: matchesTotal > matches.length, limits,
+    external_firewall: { observed_at: snapshot?.collectedAt ?? null, rule_count: externalRules, complete: false },
+  }
+}

apps/api/src/services/policy/mikrotik-rsc.test.ts

@@ -1,62 +1,82 @@
 import { describe, expect, it } from 'vitest'
-import {
+import { POLICY_APPLY_VERSION, type EvaluatedPolicy } from './evaluate.js'
-  POLICY_APPLY_VERSION,
+import { isIpv4Cidr, renderMikrotikPolicyRsc } from './mikrotik-rsc.js'
-  type EvaluatedPolicy,
-} from './evaluate.js'
-import { renderMikrotikPolicyRsc } from './mikrotik-rsc.js'
 
-function basePolicy(
+function policy(patch: Partial<EvaluatedPolicy> = {}): EvaluatedPolicy {
-  patch: Partial<EvaluatedPolicy> = {},
+  return { generation: 3, hash: 'sha256:' + 'a'.repeat(64), applyVersion: POLICY_APPLY_VERSION, defaultAction: 'accept', policyMode: 'blacklist', denyCidrs: ['1.2.3.0/24', '2001:db8::/32'], allowCidrs: ['8.8.8.8/32', '2001:db8:1::/64'], portRules: [], conflictsDropped: 0, syncIntervalSec: 60, chain: [], summary: { sets: 1, rulesDeny: 1, rulesAllow: 1, cidrsDeny: 2, cidrsAllow: 2, overrides: 0, conflictsDropped: 0, portRules: 0 }, ...patch }
-): EvaluatedPolicy {
-  return {
-    generation: 3,
-    hash: 'sha256:abc',
-    applyVersion: POLICY_APPLY_VERSION,
-    defaultAction: 'accept',
-    policyMode: 'blacklist',
-    denyCidrs: ['1.2.3.0/24', '2001:db8::/32', '10.0.0.1/32'],
-    allowCidrs: ['8.8.8.8/32', 'fe80::1/128'],
-    portRules: [],
-    conflictsDropped: 0,
-    syncIntervalSec: 60,
-    chain: [],
-    summary: {
-      sets: 1,
-      rulesDeny: 1,
-      rulesAllow: 1,
-      cidrsDeny: 2,
-      cidrsAllow: 1,
-      overrides: 0,
-      conflictsDropped: 0,
-      portRules: 0,
-    },
-    ...patch,
-  }
 }
 
-describe('renderMikrotikPolicyRsc', () => {
+describe('RouterOS dual-stack canonical compiler (rendered contract, no device runtime)', () => {
-  it('renders accept default: lists + default-drop disabled', () => {
+  it('stages both family banks and complete disabled rule candidates before activating either family', () => {
-    const rsc = renderMikrotikPolicyRsc(basePolicy())
+    const rsc = renderMikrotikPolicyRsc(policy())
-    expect(rsc).toContain(
+    expect(rsc).toContain('/ip firewall address-list add list=$deny4 address=1.2.3.0/24')
-      `# evofw hash=sha256:abc default_action=accept apply_version=${POLICY_APPLY_VERSION} gen=3`,
+    expect(rsc).toContain('/ipv6 firewall address-list add list=$deny6 address=2001:db8::/32')
-    )
+    expect(rsc).toContain('/ipv6 firewall address-list add list=$allow6 address=2001:db8:1::/64')
-    expect(rsc).toContain('list=EVOFW_DENY')
+    const firstActivation = rsc.indexOf('/ip firewall filter set [find comment=("evofw-stage-"')
-    expect(rsc).toContain('list=EVOFW_ALLOW')
+    expect(rsc.lastIndexOf(':set new6')).toBeLessThan(firstActivation)
-    expect(rsc).toContain('address=1.2.3.0/24')
+    expect(rsc.indexOf('address=2001:db8:1::/64')).toBeLessThan(firstActivation)
-    expect(rsc).not.toContain('2001:db8')
+    expect(rsc).toContain('filter add chain=input action=drop src-address-list=$deny6 disabled=yes')
-    expect(rsc).toContain(
+    expect(rsc).not.toContain('filter add chain=input action=accept')
-      'evofw-default-drop-forward] disabled=yes',
+    expect(rsc).toContain('"-default-drop-forward")] disabled=yes')
-    )
-    expect(rsc).toContain('evofw-deny-drop-input] disabled=no')
   })
-
+  it('keeps original matchers and order intact, with both-family compensating rollback and disabled-state restoration', () => {
-  it('renders drop default: default-drop enabled', () => {
+    const rsc = renderMikrotikPolicyRsc(policy({ defaultAction: 'drop' }))
-    const rsc = renderMikrotikPolicyRsc(
+    for (const [family, path] of [[4, '/ip firewall'], [6, '/ipv6 firewall']] as const) {
-      basePolicy({ defaultAction: 'drop', policyMode: 'whitelist' }),
+      expect(rsc).toContain(`${path} filter print as-value`)
-    )
+      expect(rsc).toContain(`:set old${family} ($old${family},{$row})`)
-    expect(rsc).toContain('default_action=drop')
+      expect(rsc).toContain(`${path} filter set ($row->".id") disabled=($row->"disabled") comment=($row->"comment")`)
-    expect(rsc).toContain(
+      expect(rsc).toContain(`${path} filter remove $new${family}`)
-      'evofw-default-drop-forward] disabled=no',
+      expect(rsc).toContain(`:foreach row in=$old${family}`)
-    )
+    }
+    expect(rsc).not.toContain('filter move ($row->".id")')
+    expect(rsc).not.toContain('filter remove [find comment=evofw-')
+    expect(rsc).toContain('CRITICAL: both-family rollback failed')
+    expect(rsc).toContain('"-default-drop-forward")] disabled=no')
+  })
+  it('supports missing and tampered owned rules through canonical replacement, bounds retired cleanup, and preserves foreign rules', () => {
+    const rsc = renderMikrotikPolicyRsc(policy())
+    const beforeStage = rsc.slice(0, rsc.indexOf('# Stage IPv4'))
+    expect(beforeStage).not.toContain('get [find comment=evofw-deny-drop-input]')
+    expect(beforeStage).toContain('^evofw-(deny-(hit|drop)-(input|forward)|allow-accept-forward|default-drop-forward)$')
+    expect(rsc).toContain('active retired rule needs manual recovery')
+    expect(rsc).toContain('filter remove $id')
+    expect(rsc).toContain('^EVOFW_(DENY|ALLOW)_')
+    expect(rsc).toContain('src-address-list=$listName]] = 0')
+    expect(rsc).toContain('unexpected owned matcher after repair')
+    expect(rsc).toContain('owned filter order drift after repair')
+    expect(rsc).not.toContain('filter remove [find]')
+  })
+  it('probes IPv6 before any mutation and reports unsupported when required; IPv4-only accept remains usable', () => {
+    const dual = renderMikrotikPolicyRsc(policy())
+    expect(dual.indexOf('IPv6 enforcement unavailable')).toBeLessThan(dual.indexOf('filter remove $id'))
+    expect(dual).toContain(':set EvofwApplyErrorCode "UNSUPPORTED_CAPABILITY"')
+    const ipv4 = renderMikrotikPolicyRsc(policy({ denyCidrs: ['192.0.2.1/32'], allowCidrs: [] }))
+    expect(ipv4).not.toContain(':error "IPv6 enforcement unavailable"')
+    expect(ipv4).toContain(':if ($v6Enabled) do={')
+  })
+  it('publishes the rendered hash/revision only after both-family verification succeeds', () => {
+    const rsc = renderMikrotikPolicyRsc(policy())
+    const hash = rsc.indexOf(':set EvofwRenderedHash "sha256:')
+    expect(hash).toBeGreaterThan(rsc.lastIndexOf('owned filter order drift after repair'))
+    expect(hash).toBeGreaterThan(rsc.lastIndexOf('both-family rollback completed'))
+    expect(rsc).toContain(':set EvofwRenderedRevision 3')
+    expect(rsc).toContain(':set EvofwRenderedIpv6 $v6Enabled')
+  })
+  it.each(['10.0.0.1; /system reboot', '2001:db8::/129', '10.0.0.0/33', 'fe80::1%eth0', '1.2.3.4/1/2'])('rejects invalid or scoped policy addresses: %s', (cidr) => {
+    expect(() => renderMikrotikPolicyRsc(policy({ denyCidrs: [cidr] }))).toThrow()
+  })
+  it('keeps the IPv4 classification helper strict', () => {
+    expect(isIpv4Cidr('192.0.2.1/32')).toBe(true)
+    expect(isIpv4Cidr('2001:db8::/64')).toBe(false)
+    expect(isIpv4Cidr('999.1.1.1')).toBe(false)
+  })
+  it('rejects unsupported Port ACL before generating a partially enforced script', () => {
+    expect(() => renderMikrotikPolicyRsc(policy({ portRules: [{ id: 'x', action: 'open', protocol: 'tcp', portStart: 443, portEnd: 443, srcCidrs: ['::/0'] }] }))).toThrow('Port ACL is unsupported')
+  })
+  it('counts canonical unique memberships before any adjacent or overlapping kernel compression', () => {
+    const entries = Array.from({ length: 50000 }, (_, i) => `10.${i >>> 16}.${(i >>> 8) & 255}.${i & 255}/32`)
+    expect(() => renderMikrotikPolicyRsc(policy({ denyCidrs: [...entries, '10.0.0.0/8'], allowCidrs: [] }))).toThrowError(expect.objectContaining({ code: 'POLICY_BUDGET_EXCEEDED' }))
+    const rsc = renderMikrotikPolicyRsc(policy({ denyCidrs: Array(100000).fill('192.0.2.129/24'), allowCidrs: [] }))
+    expect(rsc.match(/address=192\.0\.2\.0\/24/g)).toHaveLength(1)
   })
 })

apps/api/src/services/policy/mikrotik-rsc.ts

@@ -1,59 +1,177 @@
+import { isIP } from 'node:net'
+import { canonicalCidr } from '@evofw/shared'
 import type { EvaluatedPolicy } from './evaluate.js'
 
-/** Skip IPv6 (contains ':') — same as Linux agent. */
+type Family = 4 | 6
+const OWNED = '^evofw-(deny-(hit|drop)-(input|forward)|allow-accept-forward|default-drop-forward)$'
+const SPECS = [
+  ['deny-drop-input', 'input', 'drop', 'deny'],
+  ['deny-drop-forward', 'forward', 'drop', 'deny'],
+  ['allow-accept-forward', 'forward', 'accept', 'allow'],
+  ['default-drop-forward', 'forward', 'drop', null],
+] as const
+
 export function isIpv4Cidr(cidr: string): boolean {
-  const t = cidr.trim()
+  try { return addressFamily(cidr) === 4 } catch { return false }
-  return t.length > 0 && !t.includes(':')
 }
 
-function escAddress(cidr: string): string {
+function addressFamily(cidr: string): Family {
-  const t = cidr.trim()
+  const [address, prefix, ...rest] = cidr.trim().split('/')
-  if (/^[0-9./-]+$/.test(t)) return t
+  const family = isIP(address)
-  return `"${t.replace(/\\/g, '\\\\').replace(/"/g, '\\"')}"`
+  if (address.includes('%') || !family || rest.length || (prefix !== undefined && (!/^\d{1,3}$/.test(prefix) || Number(prefix) > (family === 4 ? 32 : 128)))) throw new Error('Invalid CIDR in MikroTik policy')
+  return family as Family
 }
 
-/**
+function familyPath(family: Family) { return family === 4 ? '/ip firewall' : '/ipv6 firewall' }
- * RouterOS 7.x script: rebuild EVOFW_* address-lists.
- * Unified chain: deny drop → allow accept → default (accept|drop).
- * Expects permanent filter rules with comments:
- *   evofw-deny-drop-input / evofw-deny-drop-forward (always on)
- *   evofw-allow-accept-forward (always on for forward path)
- *   evofw-default-drop-forward (enabled when default_action=drop)
- */
-export function renderMikrotikPolicyRsc(policy: EvaluatedPolicy): string {
-  const defaultDrop = policy.defaultAction === 'drop'
-  const defaultDropDisabled = defaultDrop ? 'no' : 'yes'
 
-  const lines: string[] = [
+/** Build canonical owned rules alongside intact originals. Rollback restores only
-    `# evofw hash=${policy.hash} default_action=${policy.defaultAction} apply_version=${policy.applyVersion} gen=${policy.generation}`,
+ * fields changed on original rules (comment/disabled); their matchers and order
-    '/ip firewall address-list remove [find list=EVOFW_DENY]',
+ * are preserved because originals are never reconstructed or moved. */
-    '/ip firewall address-list remove [find list=EVOFW_ALLOW]',
+function stageFamily(policy: EvaluatedPolicy, family: Family): string[] {
+  const path = familyPath(family)
+  const lines = [`# Stage IPv${family} banks; old referenced lists stay intact.`]
+  for (const [kind, cidrs] of [['deny', policy.denyCidrs], ['allow', policy.allowCidrs]] as const) {
+    for (const cidr of cidrs) if (addressFamily(cidr) === family) lines.push(`${path} address-list add list=$${kind}${family} address=${cidr.trim()} comment=evofw`)
+  }
+  // Add in reverse order and move each disabled rule to the front. Per-chain
+  // ordering becomes deny -> allow -> default while foreign rules stay intact.
+  for (const [name, chain, action, kind] of [...SPECS].reverse()) {
+    const comment = `("evofw-stage-" . $tx . "-${name}")`
+    const source = kind ? ` src-address-list=$${kind}${family}` : ''
+    lines.push(
+      `${path} filter add chain=${chain} action=${action}${source} disabled=yes comment=${comment}`,
+      `:local made [${path} filter find comment=${comment}]`,
+      `:if ([:len $made] != 1) do={ :error "owned staged rule missing or duplicated" }`,
+      `:set new${family} ($new${family},$made)`,
+      `:local first [${path} filter find where dynamic=no]`,
+      `:if (($first->0) != ($made->0)) do={ ${path} filter move $made destination=($first->0) }`,
+    )
+  }
+  return lines
+}
+
+function snapshotFamily(family: Family): string[] {
+  const path = familyPath(family)
+  return [
+    // Prior successful transaction's retired rules are disabled and disposable.
+    `:foreach id in=[${path} filter find where comment~"^evofw-retired-"] do={`,
+    `  :if ([${path} filter get $id disabled] != true) do={ :error "active retired rule needs manual recovery" }`,
+    `  ${path} filter remove $id`,
+    '}',
+    // Reap only owned unreferenced banks, using an immutable print snapshot.
+    `:foreach row in=[${path} address-list print as-value where list~"^EVOFW_(DENY|ALLOW)_"] do={`,
+    '  :local listName ($row->"list")',
+    `  :if ([:len [${path} filter find where src-address-list=$listName]] = 0) do={ ${path} address-list remove [find list=$listName] }`,
+    '}',
+    `:foreach row in=[${path} filter print as-value] do={`,
+    '  :local comment ($row->"comment")',
+    `  :if ($comment~"${OWNED}" || $comment~"^evofw-stage-") do={ :set old${family} ($old${family},{$row}) }`,
+    '}',
   ]
-
-  for (const c of policy.denyCidrs) {
-    if (!isIpv4Cidr(c)) continue
-    lines.push(
-      `/ip firewall address-list add list=EVOFW_DENY address=${escAddress(c)} comment=evofw`,
-    )
-  }
-  for (const c of policy.allowCidrs) {
-    if (!isIpv4Cidr(c)) continue
-    lines.push(
-      `/ip firewall address-list add list=EVOFW_ALLOW address=${escAddress(c)} comment=evofw`,
-    )
 }
 
+function activateFamily(policy: EvaluatedPolicy, family: Family): string[] {
+  const path = familyPath(family)
+  const lines: string[] = []
+  for (const [name] of SPECS) {
+    const disabled = name === 'default-drop-forward' && policy.defaultAction === 'accept' ? 'yes' : 'no'
+    lines.push(`${path} filter set [find comment=("evofw-stage-" . $tx . "-${name}")] disabled=${disabled}`)
+  }
+  // Both family candidates are complete before any old protection is retired.
   lines.push(
-    `:do { /ip firewall filter set [find comment=evofw-deny-drop-input] disabled=no } on-error={}`,
+    `:foreach row in=$old${family} do={`,
-    `:do { /ip firewall filter set [find comment=evofw-deny-drop-forward] disabled=no } on-error={}`,
+    `  ${path} filter set ($row->".id") disabled=yes comment=("evofw-retired-" . $tx . "-" . ($row->"comment"))`,
-    `:do { /ip firewall filter set [find comment=evofw-allow-accept-forward] disabled=no } on-error={}`,
+    '}',
-    `:do { /ip firewall filter set [find comment=evofw-default-drop-forward] disabled=${defaultDropDisabled} } on-error={}`,
-    // Legacy comments from bl/wl toggle era — keep disabled
-    `:do { /ip firewall filter set [find comment=evofw-bl-drop-input] disabled=yes } on-error={}`,
-    `:do { /ip firewall filter set [find comment=evofw-bl-drop-forward] disabled=yes } on-error={}`,
-    `:do { /ip firewall filter set [find comment=evofw-wl-accept-forward] disabled=yes } on-error={}`,
-    `:do { /ip firewall filter set [find comment=evofw-wl-drop-forward] disabled=yes } on-error={}`,
   )
+  for (const [name] of SPECS) lines.push(`${path} filter set [find comment=("evofw-stage-" . $tx . "-${name}")] comment=evofw-${name}`)
+  return lines
+}
 
-  return `${lines.join('\n')}\n`
+function verifyFamily(policy: EvaluatedPolicy, family: Family): string[] {
+  const path = familyPath(family)
+  const lines: string[] = []
+  for (const [name, chain, action, kind] of SPECS) {
+    const disabled = name === 'default-drop-forward' && policy.defaultAction === 'accept' ? 'true' : 'false'
+    lines.push(
+      `:local id [${path} filter find comment=evofw-${name}]`,
+      `:if ([:len $id] != 1 || [${path} filter get $id chain] != "${chain}" || [${path} filter get $id action] != "${action}" || [${path} filter get $id disabled] != ${disabled}) do={ :error "owned filter chain/action drift after repair" }`,
+    )
+    if (kind) lines.push(`:if ([${path} filter get $id src-address-list] != $${kind}${family}) do={ :error "owned source bank drift after repair" }`)
+    else lines.push(`:if ([${path} filter get $id src-address-list] != "") do={ :error "owned default matcher drift after repair" }`)
+    lines.push(
+      ':local permitted {".id";"chain";"action";"src-address-list";"disabled";"dynamic";"invalid";"bytes";"packets";"comment";"log";"log-prefix"}',
+      `:foreach row in=[${path} filter print as-value where comment=evofw-${name}] do={`,
+      '  :if (($row->"invalid") = true || ($row->"dynamic") = true) do={ :error "owned rule is not active static configuration" }',
+      '  :foreach key,value in=$row do={ :if ([:typeof [:find $permitted $key]] = "nil") do={ :error "unexpected owned matcher after repair" } }',
+      '}',
+    )
+  }
+  for (const chain of ['input', 'forward'] as const) {
+    lines.push(`:local ordered [${path} filter find where chain=${chain} and dynamic=no]`)
+    SPECS.filter((spec) => spec[1] === chain).forEach(([name], index) => lines.push(`:if (($ordered->${index}) != ([${path} filter find comment=evofw-${name}]->0)) do={ :error "owned filter order drift after repair" }`))
+  }
+  return lines
+}
+
+function rollbackFamily(family: Family): string[] {
+  const path = familyPath(family)
+  return [
+    ':do {',
+    `  :foreach row in=$old${family} do={ ${path} filter set ($row->".id") disabled=($row->"disabled") comment=($row->"comment") }`,
+    `  ${path} filter remove $new${family}`,
+    '} on-error={ :set rollbackFailed true }',
+  ]
+}
+
+/** RouterOS has no cross-family transaction: preserve full original rule
+ * snapshots and compensate both families if any stage/activation/verify fails. */
+export function renderMikrotikPolicyRsc(policy: EvaluatedPolicy): string {
+  if (!/^sha256:[0-9a-f]{64}$/.test(policy.hash) || !Number.isSafeInteger(policy.generation) || policy.generation < 0 || !Number.isSafeInteger(policy.applyVersion) || !['accept', 'drop'].includes(policy.defaultAction)) throw new Error('Invalid MikroTik policy metadata')
+  const rawCount = policy.denyCidrs.length + policy.allowCidrs.length + policy.portRules.reduce((n, rule) => n + rule.srcCidrs.length, 0)
+  if (rawCount > 200000 || policy.portRules.length > 1000 || Buffer.byteLength(JSON.stringify(policy), 'utf8') > 8388608) throw Object.assign(new Error('Raw policy budget exceeded'), { code: 'POLICY_BUDGET_EXCEEDED' })
+  const normalize = (cidrs: string[]) => [...new Set(cidrs.map((cidr) => {
+    if (typeof cidr !== 'string' || cidr.length > 128) throw new Error('Invalid CIDR in MikroTik policy')
+    return canonicalCidr(cidr)
+  }))]
+  policy = { ...policy, denyCidrs: normalize(policy.denyCidrs), allowCidrs: normalize(policy.allowCidrs), portRules: policy.portRules.map((rule) => ({ ...rule, srcCidrs: normalize(rule.srcCidrs) })) }
+  const effectiveCount = policy.denyCidrs.length + policy.allowCidrs.length + policy.portRules.reduce((n, rule) => n + new Set(rule.srcCidrs).size, 0)
+  if (effectiveCount > 50000 || Buffer.byteLength(JSON.stringify(policy), 'utf8') > 8388608) throw Object.assign(new Error('Effective policy budget exceeded'), { code: 'POLICY_BUDGET_EXCEEDED' })
+  for (const cidr of [...policy.denyCidrs, ...policy.allowCidrs]) addressFamily(cidr)
+  if (policy.portRules.length) throw Object.assign(new Error('MikroTik Port ACL is unsupported'), { code: 'UNSUPPORTED_CAPABILITY' })
+  const requiresIpv6 = [...policy.denyCidrs, ...policy.allowCidrs].some((cidr) => addressFamily(cidr) === 6)
+  const lines = [
+    `# evofw hash=${policy.hash} default_action=${policy.defaultAction} apply_version=${policy.applyVersion} gen=${policy.generation}`,
+    ':do {',
+    ':global EvofwRenderedHash', ':global EvofwRenderedRevision', ':global EvofwRenderedIpv6',
+    ':global EvofwApplyErrorCode', ':set EvofwApplyErrorCode "apply_failed"',
+    ':local v6Enabled false',
+    ':do { :set v6Enabled (![/ipv6 settings get disable-ipv6]) } on-error={ :set v6Enabled false }',
+    ...(requiresIpv6 ? [':if (!$v6Enabled) do={ :set EvofwApplyErrorCode "UNSUPPORTED_CAPABILITY"; :error "IPv6 enforcement unavailable" }'] : []),
+    ':local tx [:rndstr length=12 from="0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"]',
+    ':local deny4 ("EVOFW_DENY_" . $tx)', ':local allow4 ("EVOFW_ALLOW_" . $tx)',
+    ':local deny6 ("EVOFW_DENY_" . $tx)', ':local allow6 ("EVOFW_ALLOW_" . $tx)',
+    ':local old4 {}', ':local old6 {}', ':local new4 {}', ':local new6 {}',
+    ...snapshotFamily(4),
+    ':if ($v6Enabled) do={', ...snapshotFamily(6), '}',
+    ':do {',
+    ...stageFamily(policy, 4),
+    ':if ($v6Enabled) do={', ...stageFamily(policy, 6), '}',
+    ...activateFamily(policy, 4),
+    ':if ($v6Enabled) do={', ...activateFamily(policy, 6), '}',
+    ...verifyFamily(policy, 4),
+    ':if ($v6Enabled) do={', ...verifyFamily(policy, 6), '}',
+    '} on-error={',
+    '  :local rollbackFailed false', ...rollbackFamily(4),
+    '  :if ($v6Enabled) do={', ...rollbackFamily(6), '  }',
+    '  :if ($rollbackFailed) do={ :set EvofwApplyErrorCode "rollback_failed"; :error "CRITICAL: both-family rollback failed" }',
+    '  :error "both-family rollback completed; policy not acknowledged"',
+    '}',
+    // Publish the actual rendered generation only after both family verifies.
+    `:set EvofwRenderedHash "${policy.hash}"`, `:set EvofwRenderedRevision ${policy.generation}`,
+    ':set EvofwRenderedIpv6 $v6Enabled',
+    '} on-error={ :error "evofw compiled policy failed; no acknowledgement" }',
+  ]
+  const result = `${lines.join('\n')}\n`
+  if (Buffer.byteLength(result, 'utf8') > 8388608) throw Object.assign(new Error('Compiled policy exceeds 8MiB'), { code: 'POLICY_BUDGET_EXCEEDED' })
+  return result
 }

apps/api/src/services/policy/resolve-hostname.ts

@@ -1,4 +1,4 @@
-import { resolve4, resolve6 } from 'node:dns/promises'
+import { Resolver } from 'node:dns/promises'
 import { createHash } from 'node:crypto'
 import type { Db } from '@evofw/db'
 import { repos } from '@evofw/db'
@@ -8,24 +8,32 @@ function hashCidrs(cidrs: string[]): string {
   return `sha256:${createHash('sha256').update(cidrs.join('\n')).digest('hex')}`
 }
 
-/** Resolve FQDN to /32 and /128 CIDRs. Throws if nothing resolved. */
+export const HOSTNAME_DNS_TIMEOUT_MS = 5000
+
+/** Publish A/AAAA together; transient failures must never drop one family. */
 export async function resolveHostnameToCidrs(hostname: string): Promise<string[]> {
   const host = hostname.trim().replace(/\.$/, '').toLowerCase()
   if (!host) throw new Error('hostname empty')
 
-  const out: string[] = []
+  const resolver = new Resolver({ timeout: HOSTNAME_DNS_TIMEOUT_MS, tries: 1 })
-  try {
+  const deadline = setTimeout(() => resolver.cancel(), HOSTNAME_DNS_TIMEOUT_MS)
-    const a = await resolve4(host)
+  deadline.unref()
-    out.push(...a.map((ip) => `${ip}/32`))
+  const family = async (query: Promise<string[]>, bits: number) => {
-  } catch {
+    try { return (await query).map(ip => `${ip}/${bits}`) }
-    /* ignore A failures */
+    catch (error) {
+      const code = (error as NodeJS.ErrnoException).code
+      // ENODATA is authoritative absence of this record family. ENOTFOUND
+      // means NXDOMAIN for the hostname, so a contradictory partial answer
+      // is not accepted. Any transient/cancelled query preserves last-good.
+      if (code === 'ENODATA') return []
+      throw new Error(`DNS resolve failed for ${host}: ${code || 'query failed'}`)
     }
-  try {
-    const aaaa = await resolve6(host)
-    out.push(...aaaa.map((ip) => `${ip}/128`))
-  } catch {
-    /* ignore AAAA failures */
   }
+  let out: string[]
+  try {
+    const [a, aaaa] = await Promise.all([family(resolver.resolve4(host), 32), family(resolver.resolve6(host), 128)])
+    out = [...a, ...aaaa]
+  } finally { clearTimeout(deadline); resolver.cancel() }
 
   const cidrs = uniqCidrs(out)
   if (cidrs.length === 0) {
@@ -46,14 +54,14 @@ export async function resolveAndStoreHostnameRule(
     .sort()
   const nextHash = hashCidrs(cidrs)
   const prevHash = hashCidrs(prev)
-  repos.replaceResolvedForRule(db, ruleId, cidrs)
+  if (nextHash !== prevHash) repos.replaceResolvedForRule(db, ruleId, cidrs)
   return cidrs.length && nextHash !== prevHash ? cidrs : cidrs
 }
 
 /** Re-resolve all hostname rules; bump agents when cache changes. */
 export async function refreshAllHostnameRules(db: Db): Promise<void> {
   const rules = repos.listHostnameRules(db)
-  const changedSets = new Set<string>()
+  const staged: { rule: typeof rules[number]; previous: string; cidrs: string[] }[] = []
 
   for (const rule of rules) {
     if (!rule.hostname) continue
@@ -63,14 +71,25 @@ export async function refreshAllHostnameRules(db: Db): Promise<void> {
       )
       const cidrs = await resolveHostnameToCidrs(rule.hostname)
       const next = hashCidrs(cidrs)
-      repos.replaceResolvedForRule(db, rule.id, cidrs)
+      if (prev !== next) staged.push({ rule, previous: prev, cidrs })
-      if (prev !== next) changedSets.add(rule.setId)
     } catch {
       /* keep previous cache on transient DNS failure */
     }
   }
 
-  for (const setId of changedSets) {
+  db.transaction((tx) => {
-    repos.bumpAgentsForSet(db, setId)
+    const changedSets = new Set<string>()
+    for (const { rule, previous, cidrs } of staged) {
+      const current = repos.getPolicyRule(tx, rule.id)
+      // A rule/cache may have been edited or another refresh published while DNS
+      // was pending. Never overwrite that newer publication with a stale lookup.
+      if (!current || current.hostname !== rule.hostname || current.setId !== rule.setId) continue
+      const currentHash = hashCidrs(repos.listResolvedForRule(tx, rule.id).map((r) => r.cidr))
+      if (currentHash !== previous) continue
+      repos.replaceResolvedForRule(tx, rule.id, cidrs)
+      changedSets.add(rule.setId)
     }
+    const alreadyBumped = new Set<string>()
+    for (const setId of changedSets) repos.bumpAgentsForSet(tx, setId, alreadyBumped)
+  })
 }

apps/api/src/services/policy/rollouts.test.ts

@@ -0,0 +1,212 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { createPolicySnapshot, startPolicyRollout, confirmPolicyRollout, rollbackPolicyRollout, releaseAgentPolicyPin, diffPolicySnapshot, getPolicyRolloutView } from './rollouts.js'
+import { evaluateAgentPolicy } from './evaluate.js'
+import { explainAgentPolicy } from './explain.js'
+import { buildApp } from '../../app.js'
+import { loadConfig } from '../../config.js'
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+afterEach(() => { vi.restoreAllMocks(); for (const mem of opened.splice(0)) mem.sqlite.close() })
+const start = '2026-01-01T00:00:00.000Z'
+const later = '2026-01-01T00:00:01.000Z'
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  for (const id of ['source', 'a', 'b', 'outside']) repos.insertAgent(mem.db, { id, name: id, tokenPrefix: id, tokenHash: id,
+    status: 'approved', telemetryVersion: 2, capabilitiesJson: '["policy-ipv4","policy-ipv6","policy-port-acl","policy-port-acl-ipv6"]' })
+  repos.insertPolicySet(mem.db, { id: 'candidate', name: 'Candidate' })
+  repos.insertPolicyRule(mem.db, { id: 'deny', setId: 'candidate', action: 'deny', cidr: '192.0.2.0/24', priority: 10 })
+  repos.setAgentPolicySets(mem.db, 'source', ['candidate'])
+  return mem
+}
+function begin(db: ReturnType<typeof memory>['db']) {
+  const snapshot = createPolicySnapshot(db, 'source', 'Candidate', start)
+  const rollout = startPolicyRollout(db, { snapshot_id: snapshot.id,
+    targets: ['a', 'b'].map((id) => ({ agent_id: id, expected_generation: repos.getAgent(db, id)!.policyGeneration })), canary_ids: ['a'] }, start)
+  return { snapshot, rollout }
+}
+function ack(db: ReturnType<typeof memory>['db'], id: string, now = later) {
+  const policy = evaluateAgentPolicy(db, id, now)
+  repos.updateAgent(db, id, { appliedHash: policy.hash, appliedRevision: policy.generation, lastApplyStatus: 'ok', lastApplyError: null, applyErrorJson: null, lastTelemetryAt: now })
+}
+
+describe('immutable policy snapshots and staged rollout', () => {
+  it('allows a policy reader to preview snapshot diffs while rejecting snapshot creation and rollout mutations', async () => {
+    const config = { ...loadConfig(), jwtSecret: 'snapshot-rbac-test-secret', authRequired: true, staticDir: null, logLevel: 'error' as const }
+    const app = await buildApp({ memory: true, config })
+    try {
+      await app.ready()
+      repos.insertAgent(app.db, { id: 'source', name: 'Source', tokenHash: 'source', tokenPrefix: 'source', status: 'approved' })
+      const snapshot = createPolicySnapshot(app.db, 'source', 'Read preview')
+      const headers = { authorization: `Bearer ${app.jwt.sign({ sub: 'reader', apps: ['fw'], permissions: ['fw:policies:read'], iss: config.authIssuer })}` }
+      expect((await app.inject({ method: 'POST', url: `/api/v1/policies/snapshots/${snapshot.id}/diff`, headers, payload: { agent_ids: ['source'] } })).statusCode).toBe(200)
+      expect((await app.inject({ method: 'POST', url: '/api/v1/policies/snapshots', headers, payload: { source_agent_id: 'source', name: 'Denied' } })).statusCode).toBe(403)
+      expect((await app.inject({ method: 'POST', url: '/api/v1/policies/rollouts', headers, payload: {} })).statusCode).toBe(403)
+      expect((await app.inject({ method: 'POST', url: '/api/v1/policies/rollouts/example/confirm', headers, payload: { expected_version: 1 } })).statusCode).toBe(403)
+      expect((await app.inject({ method: 'DELETE', url: `/api/v1/policies/snapshots/${snapshot.id}`, headers })).statusCode).toBe(403)
+    } finally { await app.close() }
+  })
+  it('terminates and cleans up 20 released-canary abandonments without repinning manually released targets', () => {
+    const { db } = memory()
+    const snapshot = createPolicySnapshot(db, 'source', 'Candidate', start)
+    const abandoned = []
+    for (let index = 0; index < 20; index++) {
+      const generation = repos.getAgent(db, 'a')!.policyGeneration
+      const rollout = startPolicyRollout(db, { snapshot_id: snapshot.id, targets: [{ agent_id: 'a', expected_generation: generation }, { agent_id: 'b', expected_generation: 1 }], canary_ids: ['a'] }, start)
+      releaseAgentPolicyPin(db, 'a', rollout.id, repos.getAgent(db, 'a')!.policyGeneration, later)
+      abandoned.push(rollout)
+    }
+    expect(repos.countActivePolicyRollouts(db)).toBe(20)
+    const releasedRevision = repos.getAgent(db, 'a')!.policyGeneration
+    for (const rollout of abandoned) {
+      const result = rollbackPolicyRollout(db, rollout.id, 1, later)
+      expect(result.state).toBe('rolled_back')
+      expect(result.targets.map((target) => target.phase)).toEqual(['released', 'pending'])
+      expect(repos.deletePolicyRollout(db, rollout.id)).toBe(1)
+    }
+    expect(repos.getPolicyPin(db, 'a')).toBeUndefined()
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(releasedRevision)
+    expect(repos.getAgent(db, 'b')!.policyGeneration).toBe(1)
+    expect(repos.countActivePolicyRollouts(db)).toBe(0)
+    expect(repos.countPolicyRollouts(db)).toBe(0)
+    expect(repos.deletePolicySnapshot(db, snapshot.id)).toBe(1)
+  })
+  it('rejects retained rollout capacity without creating snapshots, targets or partial pins', () => {
+    const { db, sqlite } = memory()
+    const snapshot = createPolicySnapshot(db, 'source', 'Candidate', start)
+    const insert = sqlite.prepare("INSERT INTO policy_rollouts VALUES (?,?,'rolled_back',2,?,?)")
+    for (let index = 0; index < 200; index++) insert.run(`retained-${index}`, snapshot.id, start, start)
+    expect(() => startPolicyRollout(db, { snapshot_id: snapshot.id, targets: [{ agent_id: 'a', expected_generation: 1 }], canary_ids: ['a'] }, start)).toThrow('capacity')
+    expect(repos.getPolicyPin(db, 'a')).toBeUndefined()
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(1)
+    expect(repos.policySnapshotCapacity(db).count).toBe(1)
+    expect(sqlite.prepare('SELECT count(*) AS n FROM policy_rollout_targets').get()).toEqual({ n: 0 })
+  })
+  it('pins canaries only, freezes source changes, confirms exact fresh ACKs, rolls back previous desired with new revisions', () => {
+    const { db } = memory()
+    const previous = evaluateAgentPolicy(db, 'a', start)
+    const { snapshot, rollout } = begin(db)
+    const canary = evaluateAgentPolicy(db, 'a', start)
+    expect(canary.hash).toBe(snapshot.hash)
+    expect(repos.getAgent(db, 'a')!.appliedHash).toBeNull()
+    expect(repos.getAgent(db, 'a')!.desiredHash).toBe(snapshot.hash)
+    expect(repos.getPolicyPin(db, 'b')).toBeUndefined()
+    expect(evaluateAgentPolicy(db, 'outside', start).generation).toBe(1)
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, later)).toThrow('acknowledgement')
+    repos.updatePolicyRule(db, 'deny', { cidr: '198.51.100.0/24' })
+    repos.bumpAgentsForSet(db, 'candidate')
+    expect(evaluateAgentPolicy(db, 'a', later)).toEqual({ ...canary })
+    expect(evaluateAgentPolicy(db, 'source', later).hash).not.toBe(snapshot.hash)
+    const explanation = explainAgentPolicy(db, 'a', { ip: '192.0.2.1', chain: 'input', connection_state: 'new', dnat: false }, later)
+    expect(explanation.desired.decision).toBe('drop')
+    expect(explanation.desired.match!.label).toContain('Frozen policy snapshot')
+    ack(db, 'a')
+    const confirmed = confirmPolicyRollout(db, rollout.id, 1, later)
+    expect(confirmed.state).toBe('confirmed')
+    expect(evaluateAgentPolicy(db, 'b', later).hash).toBe(snapshot.hash)
+    expect(repos.getAgent(db, 'b')!.appliedHash).toBeNull()
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, later)).toThrow('state changed')
+    const rolledBack = rollbackPolicyRollout(db, rollout.id, 2, later)
+    expect(rolledBack.state).toBe('rolled_back')
+    expect(evaluateAgentPolicy(db, 'a', later).hash).toBe(previous.hash)
+    expect(evaluateAgentPolicy(db, 'a', later).generation).toBeGreaterThan(canary.generation)
+    expect(repos.getAgent(db, 'a')!.appliedHash).toBe(snapshot.hash)
+    expect(rolledBack.targets.every((target) => !target.acknowledged)).toBe(true)
+    const released = releaseAgentPolicyPin(db, 'a', rollout.id, evaluateAgentPolicy(db, 'a', later).generation, later)
+    expect(released.generation).toBeGreaterThan(canary.generation)
+    expect(repos.getPolicyPin(db, 'a')).toBeUndefined()
+  })
+
+  it('rejects live TTL overrides on snapshot source and every rollback baseline before any publication', () => {
+    const { db, sqlite } = memory()
+    repos.insertOverride(db, { id: 'ttl', agentId: 'source', action: 'allow', cidr: '203.0.113.1/32', expiresAt: '2026-01-02T00:00:00.000Z' })
+    expect(() => createPolicySnapshot(db, 'source', 'Candidate', start)).toThrow('temporary override')
+    repos.deleteOverride(db, 'ttl')
+    const snapshot = createPolicySnapshot(db, 'source', 'Candidate', start)
+    repos.insertOverride(db, { id: 'ttl2', agentId: 'b', action: 'deny', cidr: '203.0.113.1/32', expiresAt: '2026-01-02T00:00:00.000Z' })
+    expect(() => startPolicyRollout(db, { snapshot_id: snapshot.id, targets: [{ agent_id: 'a', expected_generation: 1 }, { agent_id: 'b', expected_generation: 1 }], canary_ids: ['a'] }, start)).toThrow('temporary override')
+    expect(repos.getPolicyPin(db, 'a')).toBeUndefined()
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(1)
+    expect(sqlite.prepare('SELECT count(*) AS n FROM policy_rollouts').get()).toEqual({ n: 0 })
+  })
+
+  it('blocks policy mutations, exposes bounded diff, and refuses unsupported targets atomically', () => {
+    const { db } = memory()
+    const { snapshot } = begin(db)
+    expect(() => repos.setAgentPolicySets(db, 'a', ['candidate'])).toThrow('Release')
+    expect(() => repos.insertOverride(db, { id: 'x', agentId: 'a', cidr: '10.0.0.0/8', action: 'allow' })).toThrow('Release')
+    expect(() => repos.updateAgent(db, 'a', { defaultAction: 'drop' })).toThrow('Release')
+    expect(() => repos.insertAgentPortRule(db, { id: 'port', agentId: 'a', action: 'open', protocol: 'tcp', portStart: 22, portEnd: 22, srcKind: 'cidr', srcCidr: '0.0.0.0/0' })).toThrow('Release')
+    expect(() => repos.deleteAgent(db, 'b')).toThrow('Release')
+    const diff = diffPolicySnapshot(db, snapshot.id, ['a', 'b', 'outside'], later)
+    expect(diff.items[0]!.conflict).toBe('POLICY_PINNED')
+    expect(diff.items[1]!.added).toBe(1)
+    expect(diff.items.reduce((total, item) => total + item.examples.length, 0)).toBeLessThanOrEqual(100)
+    repos.updateAgent(db, 'outside', { telemetryVersion: 1 })
+    expect(() => startPolicyRollout(db, { snapshot_id: snapshot.id, targets: [{ agent_id: 'outside', expected_generation: 1 }], canary_ids: ['outside'] }, later)).toThrow('capabilities')
+    expect(repos.getPolicyPin(db, 'outside')).toBeUndefined()
+  })
+
+  it('checks canary activation time, freshness, exact revision, failure and pending generation at confirm', () => {
+    const { db } = memory()
+    const { rollout, snapshot } = begin(db)
+    ack(db, 'a', start)
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, later)).toThrow('acknowledgement')
+    ack(db, 'a')
+    repos.updateAgent(db, 'a', { appliedRevision: 1 })
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, later)).toThrow('acknowledgement')
+    ack(db, 'a')
+    repos.updateAgent(db, 'a', { lastApplyStatus: 'error' })
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, later)).toThrow('acknowledgement')
+    ack(db, 'a')
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, '2026-01-01T00:10:00.000Z')).toThrow('acknowledgement')
+    repos.bumpAgentGeneration(db, 'b')
+    expect(() => confirmPolicyRollout(db, rollout.id, 1, later)).toThrow('generation changed')
+    expect(repos.getPolicyPin(db, 'b')).toBeUndefined()
+    expect(getPolicyRolloutView(db, rollout.id, later).version).toBe(1)
+    expect(repos.getAgent(db, 'a')!.desiredHash).toBe(snapshot.hash)
+  })
+
+  it('deduplicates snapshots, enforces global storage capacity transactionally and keeps a pin if live recompile fails', () => {
+    const { db, sqlite } = memory()
+    const { rollout, snapshot } = begin(db)
+    expect(createPolicySnapshot(db, 'source', 'Duplicate label', later).id).toBe(snapshot.id)
+    // Capacity uses actual UTF-8 bytes, independently of the denormalized metadata.
+    sqlite.prepare('UPDATE policy_snapshots SET payload_bytes=0').run()
+    const actualBytes = sqlite.prepare('SELECT sum(length(CAST(payload_json AS BLOB))) AS n FROM policy_snapshots').get() as { n: number }
+    expect(repos.policySnapshotCapacity(db).bytes).toBe(actualBytes.n)
+    const capacity = vi.spyOn(repos, 'policySnapshotCapacity').mockReturnValue({ count: 2, bytes: 256 * 1024 * 1024 })
+    repos.insertOverride(db, { id: 'permanent', agentId: 'outside', cidr: '10.0.0.0/8', action: 'allow' })
+    expect(() => createPolicySnapshot(db, 'outside', 'No capacity', later)).toThrow('budget')
+    capacity.mockRestore()
+    repos.insertPolicySet(db, { id: 'bad', name: 'Bad' })
+    repos.setAgentPolicySets(db, 'source', ['bad'])
+    // Target live baseline becomes incompatible while the immutable candidate remains valid.
+    sqlite.prepare('UPDATE agents SET default_action = ? WHERE id = ?').run('drop', 'a')
+    repos.updateAgent(db, 'a', { capabilitiesJson: '[]' })
+    expect(() => releaseAgentPolicyPin(db, 'a', rollout.id, repos.getAgent(db, 'a')!.policyGeneration, later)).toThrow('capabilities')
+    expect(repos.getPolicyPin(db, 'a')!.snapshot_id).toBe(snapshot.id)
+    expect(repos.getAgent(db, 'a')!.policyGeneration).toBe(2)
+  })
+
+  it('enforces HTTP schemas and pin conflicts through actual endpoints', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, staticDir: null, logLevel: 'error' } })
+    try {
+      await app.ready()
+      for (const id of ['source', 'a']) repos.insertAgent(app.db, { id, name: id, tokenPrefix: id, tokenHash: id, status: 'approved', telemetryVersion: 2, capabilitiesJson: '["policy-ipv4"]' })
+      const snapshot = await app.inject({ method: 'POST', url: '/api/v1/policies/snapshots', payload: { source_agent_id: 'source', name: 'Candidate' } })
+      expect(snapshot.statusCode).toBe(201)
+      expect(snapshot.json().payload_json).toBeUndefined()
+      const invalid = await app.inject({ method: 'POST', url: '/api/v1/policies/rollouts', payload: { snapshot_id: snapshot.json().id, targets: [{ agent_id: 'a', expected_generation: 1 }], canary_ids: ['missing'] } })
+      expect(invalid.statusCode).toBe(400)
+      const rollout = await app.inject({ method: 'POST', url: '/api/v1/policies/rollouts', payload: { snapshot_id: snapshot.json().id, targets: [{ agent_id: 'a', expected_generation: 1 }], canary_ids: ['a'] } })
+      expect(rollout.statusCode).toBe(201)
+      const conflict = await app.inject({ method: 'PUT', url: '/api/v1/agents/a/policy-sets', payload: { set_ids: [] } })
+      expect(conflict.statusCode).toBe(409)
+      expect(conflict.json().error.code).toBe('POLICY_PINNED')
+      const deletion = await app.inject({ method: 'DELETE', url: `/api/v1/policies/snapshots/${snapshot.json().id}` })
+      expect(deletion.statusCode).toBe(409)
+      expect((await app.inject({ url: '/api/v1/policies/snapshots?limit=51' })).statusCode).toBe(400)
+    } finally { await app.close() }
+  })
+})

apps/api/src/services/policy/rollouts.ts

@@ -0,0 +1,166 @@
+import { createHash, randomUUID } from 'node:crypto'
+import { repos, type Db, type PolicySnapshotRow, type PolicyRolloutTargetRow } from '@evofw/db'
+import { POLICY_BUDGET, type CreatePolicyRollout, type PolicyRollout } from '@evofw/shared'
+import { AppError } from '../../plugins/error-handler.js'
+import { compileAgentPolicy, evaluateAgentPolicy, type EvaluatedPolicy } from './evaluate.js'
+
+const fail = (code: string, message: string, status = 409): never => { throw new AppError(code, message, status) }
+const requireAgent = (db: Db, id: string) => repos.getAgent(db, id) ?? fail('NOT_FOUND', 'Agent not found', 404)
+const requireSnapshot = (db: Db, id: string) => repos.getPolicySnapshot(db, id) ?? fail('NOT_FOUND', 'Snapshot not found', 404)
+type SnapshotPayload = Omit<EvaluatedPolicy, 'generation' | 'syncIntervalSec'>
+const payloadOf = (snapshot: PolicySnapshotRow): SnapshotPayload => JSON.parse(snapshot.payload_json)
+function noTemporaryOverrides(db: Db, id: string, now: string) {
+  if (repos.listActiveOverrides(db, id, now).some((row) => row.expiresAt !== null)) fail('TEMPORARY_OVERRIDE_ACTIVE', 'A temporary override must expire or be removed before snapshot or rollout')
+}
+function snapshotOf(db: Db, id: string, name: string, now: string) {
+  requireAgent(db, id)
+  if (repos.getPolicyPin(db, id)) fail('POLICY_PINNED', 'Release source policy pin before creating a snapshot')
+  noTemporaryOverrides(db, id, now)
+  const { generation, syncIntervalSec: _, ...payload } = evaluateAgentPolicy(db, id, now)
+  const json = JSON.stringify(payload)
+  const contentHash = createHash('sha256').update(json).digest('hex')
+  const same = repos.findPolicySnapshot(db, contentHash)
+  if (same) return same
+  const bytes = Buffer.byteLength(json)
+  const capacity = repos.policySnapshotCapacity(db)
+  if (bytes > POLICY_BUDGET.jsonBytes || capacity.count >= 200 || capacity.bytes + bytes > 256 * 1024 * 1024) fail('SNAPSHOT_CAPACITY', 'Snapshot count or stored payload byte budget exceeded', 422)
+  const row: PolicySnapshotRow = { id: randomUUID(), name, source_agent_id: id, source_generation: generation, hash: payload.hash, content_hash: contentHash, payload_json: json, payload_bytes: bytes, created_at: now }
+  repos.insertPolicySnapshot(db, row)
+  return row
+}
+export function createPolicySnapshot(db: Db, agentId: string, name: string, now = new Date().toISOString()) {
+  return db.transaction((tx) => snapshotOf(tx, agentId, name, now))
+}
+function compatible(db: Db, agentId: string, payload: SnapshotPayload) {
+  const agent = requireAgent(db, agentId)
+  if (agent.status !== 'approved') fail('ROLLOUT_TARGET_UNAVAILABLE', 'Every target must be approved')
+  let capabilities: string[] = []
+  try { capabilities = JSON.parse(agent.capabilitiesJson) } catch { /* unknown capabilities cannot prove support */ }
+  const ipv6 = [...payload.denyCidrs, ...payload.allowCidrs].some((cidr) => cidr.includes(':')) || payload.portRules.some((rule) => rule.srcCidrs.some((cidr) => cidr.includes(':')))
+  const ipv6Acl = payload.portRules.some((rule) => rule.srcCidrs.some((cidr) => cidr.includes(':')))
+  if (agent.telemetryVersion !== 2 || !capabilities.includes('policy-ipv4') ||
+    (ipv6 && !capabilities.includes('policy-ipv6')) ||
+    (payload.portRules.length > 0 && (agent.platform === 'mikrotik' || !capabilities.includes('policy-port-acl') || /ipset|iptables/.test(agent.lastApplyKernelMethod ?? ''))) ||
+    (ipv6Acl && !capabilities.includes('policy-port-acl-ipv6'))) fail('UNSUPPORTED_POLICY', 'Target has not reported capabilities required by this snapshot')
+}
+function unpinnedTarget(db: Db, agentId: string, expected: number, now: string, payload: SnapshotPayload) {
+  const agent = requireAgent(db, agentId)
+  if (repos.getPolicyPin(db, agentId)) fail('POLICY_PINNED', 'Target already has a policy pin')
+  // Publish expiry before checking optimistic generation, so an expired override
+  // never sneaks into the captured previous desired state.
+  repos.expireOverridesForAgent(db, agentId, now)
+  if (requireAgent(db, agentId).policyGeneration !== expected) fail('REVISION_CONFLICT', 'Target policy generation changed; refresh preview')
+  noTemporaryOverrides(db, agentId, now)
+  compatible(db, agentId, payload)
+  return agent
+}
+export function startPolicyRollout(db: Db, body: CreatePolicyRollout, now = new Date().toISOString()) {
+  return db.transaction((tx) => {
+    const snapshot = requireSnapshot(tx, body.snapshot_id)
+    const payload = payloadOf(snapshot)
+    if (repos.countActivePolicyRollouts(tx) >= 20 || repos.countPolicyRollouts(tx) >= 200) fail('ROLLOUT_CAPACITY', 'Rollout capacity exceeded (20 active canaries or 200 retained rollouts)', 422)
+    // Validate all targets before capturing or publishing any snapshot or pin.
+    for (const target of body.targets) unpinnedTarget(tx, target.agent_id, target.expected_generation, now, payload)
+    const previous = body.targets.map((target) => snapshotOf(tx, target.agent_id, 'Previous desired policy', now))
+    const id = randomUUID()
+    repos.insertPolicyRollout(tx, { id, snapshot_id: snapshot.id, state: 'canary', version: 1, created_at: now, updated_at: now })
+    for (const [index, target] of body.targets.entries()) {
+      const agent = requireAgent(tx, target.agent_id)
+      const canary = body.canary_ids.includes(target.agent_id)
+      repos.insertPolicyRolloutTarget(tx, { rollout_id: id, agent_id: agent.id, previous_snapshot_id: previous[index]!.id,
+        baseline_generation: agent.policyGeneration, previous_delivered_hash: agent.lastPolicyDeliveredAt ? agent.desiredHash : null, previous_delivered_revision: agent.lastPolicyDeliveredAt ? agent.desiredHashGeneration : null,
+        previous_applied_hash: agent.appliedHash, previous_applied_revision: agent.appliedRevision,
+        is_canary: Number(canary), phase: 'pending', desired_revision: null, activated_at: null })
+      if (canary) repos.activatePolicyPin(tx, id, agent.id, snapshot.id, 'candidate', now)
+    }
+    return getPolicyRolloutView(tx, id, now)
+  })
+}
+function acknowledged(db: Db, target: PolicyRolloutTargetRow, hash: string, now: string) {
+  const agent = requireAgent(db, target.agent_id)
+  const sync = Number(repos.getSetting(db, 'agent_sync_interval_sec') || '60') || 60
+  const freshness = Math.min(3_600_000, Math.max(180_000, 3 * sync * 1000))
+  const telemetry = Date.parse(agent.lastTelemetryAt ?? '')
+  const activation = Date.parse(target.activated_at ?? '')
+  return agent.appliedHash === hash && agent.appliedRevision === target.desired_revision && agent.lastApplyStatus === 'ok' && !agent.lastApplyError && !agent.applyErrorJson &&
+    Number.isFinite(telemetry) && telemetry > activation && telemetry <= Date.parse(now) && Date.parse(now) - telemetry <= freshness
+}
+export function getPolicyRolloutView(db: Db, id: string, now = new Date().toISOString()): PolicyRollout {
+  const row = repos.getPolicyRollout(db, id) ?? fail('NOT_FOUND', 'Rollout not found', 404)
+  const candidateHash = repos.getPolicySnapshotHash(db, row.snapshot_id)!
+  return { ...row, targets: repos.listPolicyRolloutTargets(db, id).map((target) => {
+    const agent = requireAgent(db, target.agent_id)
+    const hash = target.phase === 'rollback' ? repos.getPolicySnapshotHash(db, target.previous_snapshot_id)! : candidateHash
+    return { agent_id: agent.id, is_canary: !!target.is_canary, phase: target.phase, desired_revision: target.desired_revision,
+      desired_hash: target.desired_revision === null || target.phase === 'released' ? null : hash, applied_revision: agent.appliedRevision, applied_hash: agent.appliedHash,
+      acknowledged: target.phase !== 'pending' && target.phase !== 'released' && acknowledged(db, target, hash, now), previous_desired_snapshot_id: target.previous_snapshot_id }
+  }) }
+}
+function requireOwnedPin(db: Db, target: PolicyRolloutTargetRow, rolloutId: string) {
+  const pin = repos.getPolicyPin(db, target.agent_id)
+  if (pin?.rollout_id !== rolloutId || requireAgent(db, target.agent_id).policyGeneration !== target.desired_revision) fail('REVISION_CONFLICT', 'Rollout target pin or revision changed')
+}
+export function confirmPolicyRollout(db: Db, id: string, version: number, now = new Date().toISOString()) {
+  return db.transaction((tx) => {
+    const rollout = repos.getPolicyRollout(tx, id) ?? fail('NOT_FOUND', 'Rollout not found', 404)
+    if (rollout.version !== version || rollout.state !== 'canary') fail('REVISION_CONFLICT', 'Rollout state changed')
+    const snapshot = requireSnapshot(tx, rollout.snapshot_id)
+    const payload = payloadOf(snapshot)
+    const targets = repos.listPolicyRolloutTargets(tx, id)
+    for (const target of targets) {
+      if (target.is_canary) {
+        requireOwnedPin(tx, target, id)
+        compatible(tx, target.agent_id, payload)
+        if (!acknowledged(tx, target, snapshot.hash, now)) fail('CANARY_NOT_ACKNOWLEDGED', 'Every canary needs a fresh successful exact hash and revision acknowledgement')
+      } else unpinnedTarget(tx, target.agent_id, target.baseline_generation, now, payload)
+    }
+    if (!repos.transitionPolicyRollout(tx, id, version, 'confirmed', now)) fail('REVISION_CONFLICT', 'Rollout version changed')
+    for (const target of targets) if (!target.is_canary) repos.activatePolicyPin(tx, id, target.agent_id, snapshot.id, 'candidate', now)
+    return getPolicyRolloutView(tx, id, now)
+  })
+}
+export function rollbackPolicyRollout(db: Db, id: string, version: number, now = new Date().toISOString()) {
+  return db.transaction((tx) => {
+    const rollout = repos.getPolicyRollout(tx, id) ?? fail('NOT_FOUND', 'Rollout not found', 404)
+    if (rollout.version !== version || rollout.state === 'rolled_back') fail('REVISION_CONFLICT', 'Rollout state changed')
+    const targets = repos.listPolicyRolloutTargets(tx, id)
+    for (const target of targets) if (target.phase !== 'pending' && target.phase !== 'released') {
+      requireOwnedPin(tx, target, id)
+      compatible(tx, target.agent_id, payloadOf(requireSnapshot(tx, target.previous_snapshot_id)))
+    }
+    if (!repos.transitionPolicyRollout(tx, id, version, 'rolled_back', now)) fail('REVISION_CONFLICT', 'Rollout version changed')
+    for (const target of targets) if (target.phase !== 'pending' && target.phase !== 'released') repos.activatePolicyPin(tx, id, target.agent_id, target.previous_snapshot_id, 'rollback', now)
+    return getPolicyRolloutView(tx, id, now)
+  })
+}
+export function releaseAgentPolicyPin(db: Db, agentId: string, rolloutId: string, expectedGeneration: number, now = new Date().toISOString()) {
+  return db.transaction((tx) => {
+    const agent = requireAgent(tx, agentId)
+    const pin = repos.getPolicyPin(tx, agentId)
+    if (pin?.rollout_id !== rolloutId || agent.policyGeneration !== expectedGeneration) fail('REVISION_CONFLICT', 'Policy pin or revision changed')
+    // Validate live candidate first. Budget/compatibility failures keep the pin.
+    const candidate = compileAgentPolicy(tx, agentId, now)
+    compatible(tx, agentId, candidate)
+    repos.releasePolicyPin(tx, agentId, rolloutId, candidate.hash)
+    return { agent_id: agentId, generation: requireAgent(tx, agentId).policyGeneration, hash: candidate.hash }
+  })
+}
+export function diffPolicySnapshot(db: Db, id: string, agentIds: string[], now = new Date().toISOString()) {
+  const candidate = payloadOf(requireSnapshot(db, id))
+  let examplesLeft = 100
+  return { snapshot_id: id, items: agentIds.map((agentId) => {
+    const agent = requireAgent(db, agentId)
+    const before = evaluateAgentPolicy(db, agentId, now)
+    const beforeMemberships = new Set([...before.denyCidrs.map((cidr) => `deny:${cidr}`), ...before.allowCidrs.map((cidr) => `allow:${cidr}`)])
+    const afterMemberships = new Set([...candidate.denyCidrs.map((cidr) => `deny:${cidr}`), ...candidate.allowCidrs.map((cidr) => `allow:${cidr}`)])
+    const added = [...afterMemberships].filter((value) => !beforeMemberships.has(value))
+    const removed = [...beforeMemberships].filter((value) => !afterMemberships.has(value))
+    const samples = [...added.map((cidr) => ({ change: 'added', cidr })), ...removed.map((cidr) => ({ change: 'removed', cidr }))].slice(0, examplesLeft)
+    examplesLeft -= samples.length
+    let conflict: string | null = null
+    try { unpinnedTarget(db, agentId, agent.policyGeneration, now, candidate) } catch (error) { if (error instanceof AppError) conflict = error.code; else throw error }
+    return { agent_id: agentId, expected_generation: requireAgent(db, agentId).policyGeneration, before_hash: before.hash, candidate_hash: candidate.hash,
+      added: added.length, removed: removed.length, examples: samples, truncated: samples.length < added.length + removed.length,
+      default_action_changed: before.defaultAction !== candidate.defaultAction, port_rules_changed: JSON.stringify(before.portRules) !== JSON.stringify(candidate.portRules), conflict }
+  }) }
+}

apps/api/src/services/port-acl.test.ts

@@ -1,6 +1,8 @@
 import { describe, it, expect, afterAll } from 'vitest'
 import { buildApp } from '../app.js'
 import type { AppConfig } from '../config.js'
+import { repos } from '@evofw/db'
+import { linkEnrollmentCredential } from './enrollment.js'
 
 const testConfig: AppConfig = {
   databaseUrl: 'sqlite::memory:',
@@ -38,7 +40,7 @@ async function enrollApprovedLinux(
     url: '/v1/agent/enroll',
     headers: {
       'content-type': 'application/json',
-      'x-evofw-seed': 'test-seed',
+      'x-evofw-enrollment-credential': linkEnrollmentCredential(app.db, repos.getInstallLink(app.db, link.id)!, testConfig.enrollSeed),
     },
     payload: {
       name,

apps/api/src/services/row-mappers.ts

@@ -1,4 +1,32 @@
 import { repos } from '@evofw/db'
+import { telemetryQualitySchema, applyErrorSchema, type AgentDiagnostics } from '@evofw/shared'
+
+export function mapAgentDiagnostics(a: NonNullable<ReturnType<typeof repos.getAgent>>): AgentDiagnostics {
+  const parseJson = (text: string | null) => { try { return JSON.parse(text ?? 'null') as unknown } catch { return null } }
+  const parsedQuality = telemetryQualitySchema.safeParse(parseJson(a.qualityJson))
+  const quality = parsedQuality.success ? parsedQuality.data : { partial: true, truncated: false, unsupported: ['telemetry_v2'], lost_reports: 0 }
+  const parsedError = applyErrorSchema.safeParse(parseJson(a.applyErrorJson))
+  const applyError = parsedError.success ? parsedError.data : a.lastApplyError ? { code: 'AGENT_APPLY_ERROR', message: a.lastApplyError } : null
+  const rawCapabilities = parseJson(a.capabilitiesJson)
+  const capabilities = Array.isArray(rawCapabilities) ? rawCapabilities.filter((item): item is string => typeof item === 'string') : []
+  const rawDetailCounts = parseJson(a.detailCountsJson)
+  const desiredHash = a.desiredHashGeneration === a.policyGeneration ? a.desiredHash : null
+  let status: AgentDiagnostics['status'] = 'pending'
+  if (applyError?.code === 'UNSUPPORTED_CAPABILITY') status = 'unsupported'
+  else if (applyError || a.lastApplyStatus === 'error' || a.lastApplyStatus === 'failed') status = 'failed'
+  else if (a.lastApplyStatus === 'unsupported' || a.telemetryVersion !== 2 || quality.unsupported.some((feature) => /policy|ipv6/.test(feature))) status = a.lastTelemetryAt ? 'unsupported' : 'pending'
+  else if (desiredHash && desiredHash === a.appliedHash && a.policyGeneration === a.appliedRevision) status = 'in_sync'
+  else if (desiredHash && a.appliedHash && desiredHash !== a.appliedHash && a.appliedRevision === a.policyGeneration) status = 'drift'
+  else if (desiredHash && a.lastPolicyDeliveredAt && (!a.lastApplyAt || a.lastPolicyDeliveredAt > a.lastApplyAt)) status = 'applying'
+  return {
+    desired_revision: a.policyGeneration, desired_hash: desiredHash,
+    applied_revision: a.appliedRevision, applied_hash: a.appliedHash,
+    agent_version: a.clientVersion, capabilities,
+    last_contact_at: a.lastSeenAt, last_telemetry_at: a.lastTelemetryAt, last_apply_at: a.lastApplyAt,
+    apply_error: applyError, status, telemetry_version: a.telemetryVersion, quality,
+    detail_counts: rawDetailCounts && typeof rawDetailCounts === 'object' ? rawDetailCounts as AgentDiagnostics['detail_counts'] : null,
+  }
+}
 
 export function mapAgent(
   a: NonNullable<ReturnType<typeof repos.getAgent>>,
@@ -33,6 +61,17 @@ export function mapAgent(
     revoked_at: a.revokedAt,
     install_curl: opts?.installCurl ?? null,
     install_link_id: opts?.installLinkId ?? null,
+    diagnostics: mapAgentDiagnostics(a),
+  }
+}
+
+export function mapOverride(row: NonNullable<ReturnType<typeof repos.insertOverride>>, now = new Date().toISOString()) {
+  return {
+    id: row.id, agent_id: row.agentId, cidr: row.cidr, action: row.action, comment: row.comment,
+    reason: row.reason, created_by_user_id: row.createdByUserId, created_by_name: row.createdByName,
+    created_at: row.createdAt, expires_at: row.expiresAt,
+    expired: !!row.expiredAt || !!(row.expiresAt && row.expiresAt <= now),
+    expiration_enforcement: 'next_policy_sync' as const,
   }
 }
 

apps/api/src/services/safe-fetch.ts

@@ -0,0 +1,121 @@
+import { lookup } from 'node:dns/promises'
+import { request as httpRequest } from 'node:http'
+import { request as httpsRequest } from 'node:https'
+import { isIP, type LookupFunction } from 'node:net'
+
+export const SOURCE_LIMITS = { bytes: 8 * 1024 * 1024, elements: 100_000, timeoutMs: 45_000, redirects: 3, pages: 1000, totalBytes: 32 * 1024 * 1024 } as const
+
+/** Conservative public destination classification, including mapped IPv4. */
+export function isPublicSourceAddress(address: string): boolean {
+  const family = isIP(address)
+  if (family === 4) {
+    const [a, b, c] = address.split('.').map(Number)
+    return !(a === 0 || a === 10 || a === 127 || a! >= 224 ||
+      (a === 100 && b! >= 64 && b! <= 127) || (a === 169 && b === 254) ||
+      (a === 172 && b! >= 16 && b! <= 31) || (a === 192 && (b === 168 || (b === 0 && (c === 0 || c === 2)))) ||
+      (a === 198 && (b === 18 || b === 19 || (b === 51 && c === 100))) || (a === 203 && b === 0 && c === 113))
+  }
+  if (family !== 6) return false
+  // A public IPv6 source must be globally routed unicast; mapped/transition
+  // addresses and special-purpose 2001::/23 + documentation ranges fail closed.
+  const normalized = new URL(`http://[${address}]/`).hostname.slice(1, -1).toLowerCase()
+  const head = Number.parseInt(normalized.split(':')[0]!, 16)
+  if (!(head >= 0x2000 && head <= 0x3fff) || head === 0x2002 || head === 0x3fff) return false
+  if (head === 0x2001) {
+    const second = Number.parseInt(normalized.split(':')[1] || '0', 16)
+    if (second <= 0x1ff || second === 0xdb8) return false
+  }
+  return true
+}
+
+function allowedInternalOrigin(url: URL): boolean {
+  // Exact origins only: no wildcard, suffix, CIDR or user-configured list bypass.
+  const entries = (process.env.EVOFW_SOURCE_ALLOWED_ORIGINS || '').split(',').map(s => s.trim()).filter(Boolean)
+  return entries.some(entry => {
+    const allowed = new URL(entry)
+    if (allowed.username || allowed.password || allowed.pathname !== '/' || allowed.search || allowed.hash || !['http:', 'https:'].includes(allowed.protocol)) {
+      throw new Error('Invalid EVOFW_SOURCE_ALLOWED_ORIGINS entry: use exact HTTP(S) origins')
+    }
+    return allowed.origin === url.origin
+  })
+}
+
+function validateUrl(input: string | URL): URL {
+  const url = new URL(input)
+  if (!['http:', 'https:'].includes(url.protocol) || url.username || url.password) throw new Error('Source requires an HTTP(S) URL without embedded credentials')
+  url.hash = ''
+  const host = url.hostname.replace(/^\[|\]$/g, '')
+  if (isIP(host) && !allowedInternalOrigin(url) && !isPublicSourceAddress(host)) throw new Error('Source destination is not public or explicitly allowed')
+  return url
+}
+
+/** The checked address is returned to the socket itself; no second DNS lookup. */
+export function sourceLookup(url: URL, resolve = lookup): LookupFunction {
+  return (_hostname, options, callback) => {
+    void resolve(url.hostname, { all: true, verbatim: true }).then(records => {
+      if (!records.length || (!allowedInternalOrigin(url) && records.some(record => !isPublicSourceAddress(record.address)))) {
+        callback(new Error('Source DNS destination is not public or explicitly allowed'), '', 4)
+        return
+      }
+      if (options.all) callback(null, records)
+      else callback(null, records[0]!.address, records[0]!.family)
+    }).catch(() => callback(new Error('Source DNS lookup failed or policy is invalid'), '', 4))
+  }
+}
+
+interface FetchOptions { headers?: Record<string, string>; maxBytes?: number; timeoutMs?: number }
+export async function safeFetchJson(input: string | URL, options: FetchOptions = {}): Promise<{ data: unknown; bytes: number }> {
+  let url = validateUrl(input)
+  const initialOrigin = url.origin
+  let sendCredentials = true
+  const deadline = Date.now() + Math.min(options.timeoutMs ?? SOURCE_LIMITS.timeoutMs, SOURCE_LIMITS.timeoutMs)
+  const maxBytes = Math.min(options.maxBytes ?? SOURCE_LIMITS.bytes, SOURCE_LIMITS.bytes)
+  for (let redirects = 0; redirects <= SOURCE_LIMITS.redirects; redirects++) {
+    const remaining = deadline - Date.now()
+    if (remaining <= 0) throw new Error('Source request deadline exceeded')
+    const headers: Record<string, string> = { Accept: 'application/json', 'Accept-Encoding': 'identity' }
+    if (sendCredentials && url.origin === initialOrigin) Object.assign(headers, options.headers)
+    const result = await new Promise<{ location?: string; data?: unknown; bytes: number }>((resolve, reject) => {
+      const request = (url.protocol === 'https:' ? httpsRequest : httpRequest)(url, {
+        method: 'GET', headers, agent: false, maxHeaderSize: 16 * 1024,
+        lookup: sourceLookup(url), signal: AbortSignal.timeout(remaining),
+      }, response => {
+        const status = response.statusCode || 0
+        if ([301, 302, 303, 307, 308].includes(status)) {
+          const location = response.headers.location
+          response.destroy()
+          if (!location) reject(new Error('Source redirect without Location'))
+          else resolve({ location, bytes: 0 })
+          return
+        }
+        if (status < 200 || status >= 300) { response.destroy(); reject(new Error(`Source HTTP ${status}`)); return }
+        if (response.headers['content-encoding'] && response.headers['content-encoding'] !== 'identity') {
+          response.destroy(); reject(new Error('Source compressed response is not supported')); return
+        }
+        const length = Number(response.headers['content-length'])
+        if (Number.isFinite(length) && length > maxBytes) { response.destroy(); reject(new Error('Source response byte limit exceeded')); return }
+        const chunks: Buffer[] = []
+        let bytes = 0
+        response.on('data', (chunk: Buffer) => {
+          bytes += chunk.length
+          if (bytes > maxBytes) { response.destroy(); reject(new Error('Source response byte limit exceeded')); return }
+          chunks.push(chunk)
+        })
+        response.on('error', () => reject(new Error('Source response interrupted')))
+        response.on('end', () => {
+          try { resolve({ data: JSON.parse(Buffer.concat(chunks).toString('utf8')) as unknown, bytes }) }
+          catch { reject(new Error('Source response is not valid JSON')) }
+        })
+      })
+      request.on('error', () => reject(new Error('Source request failed or timed out')))
+      request.end()
+    })
+    if (!result.location) return { data: result.data, bytes: result.bytes }
+    if (redirects === SOURCE_LIMITS.redirects) throw new Error('Source redirect limit exceeded')
+    const next = validateUrl(new URL(result.location, url))
+    if (url.protocol === 'https:' && next.protocol !== 'https:') throw new Error('Source HTTPS downgrade redirect rejected')
+    if (next.origin !== url.origin) sendCredentials = false
+    url = next
+  }
+  throw new Error('Source redirect limit exceeded')
+}

apps/api/src/services/settings-secrets.test.ts

@@ -0,0 +1,31 @@
+import { describe, expect, it } from 'vitest'
+import { repos } from '@evofw/db'
+import { buildApp } from '../app.js'
+import { loadConfig } from '../config.js'
+
+describe('enrollment signing master exposure', () => {
+  it('masks configured and fallback signing masters, preserves sentinel updates and excludes values from audit', async () => {
+    const master = 'private-enrollment-signing-master'
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), enrollSeed: master, authRequired: false, authAuditIngestSecret: null, staticDir: null, logLevel: 'silent' } })
+    try {
+      const settings = await app.inject({ url: '/api/v1/settings' })
+      expect(settings.json().enroll_seed).toBe('********')
+      expect(settings.body).not.toContain(master)
+      expect(settings.headers['cache-control']).toBe('no-store')
+      const context = await app.inject({ url: '/api/v1/install-context' })
+      expect(context.json()).not.toHaveProperty('enroll_seed')
+      expect(context.json().enroll_seed_configured).toBe(true)
+      expect(context.body).not.toContain(master)
+      expect((await app.inject({ method: 'PUT', url: '/api/v1/settings', payload: { enroll_seed: '********' } })).statusCode).toBe(200)
+      expect(repos.getSetting(app.db, 'enroll_seed')).toBe(master)
+      expect((await app.inject({ method: 'PUT', url: '/api/v1/settings', payload: { enroll_seed: 'explicit-new-signing-master' } })).statusCode).toBe(200)
+      expect(repos.getSetting(app.db, 'enroll_seed')).toBe('explicit-new-signing-master')
+      const audit = await app.inject({ url: '/api/v1/audit' })
+      expect(audit.body).not.toContain(master)
+      expect(audit.body).not.toContain('explicit-new-signing-master')
+      // Empty persisted value uses the configured fallback without disclosing it.
+      repos.setSetting(app.db, 'enroll_seed', '')
+      expect((await app.inject({ url: '/api/v1/settings' })).json().enroll_seed).toBe('********')
+    } finally { await app.close() }
+  })
+})

apps/api/src/services/source-dns.test.ts

@@ -0,0 +1,45 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+const dns = vi.hoisted(() => ({ resolve4: vi.fn(), resolve6: vi.fn(), cancel: vi.fn() }))
+vi.mock('node:dns/promises', async importOriginal => ({
+  ...await importOriginal<typeof import('node:dns/promises')>(),
+  Resolver: class { resolve4 = dns.resolve4; resolve6 = dns.resolve6; cancel = dns.cancel },
+}))
+import { HOSTNAME_DNS_TIMEOUT_MS, resolveHostnameToCidrs, resolveAndStoreHostnameRule } from './policy/resolve-hostname.js'
+afterEach(() => { vi.useRealTimers(); vi.resetAllMocks() })
+describe('complete hostname snapshots', () => {
+  it('retains both cached families when AAAA times out while A changes', async () => {
+    const { db, sqlite } = createMemoryDb(); runMigrations(sqlite)
+    try {
+      repos.insertPolicySet(db, { id: 'dns-set', name: 'DNS test' })
+      repos.insertPolicyRule(db, { id: 'dns-rule', setId: 'dns-set', priority: 10, action: 'deny', hostname: 'host.test' })
+      dns.resolve4.mockResolvedValue(['1.1.1.1']); dns.resolve6.mockResolvedValue(['2606:4700:4700::1111'])
+      await resolveAndStoreHostnameRule(db, 'dns-rule', 'host.test')
+      const before = repos.listResolvedForRule(db, 'dns-rule')
+      dns.resolve4.mockResolvedValue(['8.8.8.8']); dns.resolve6.mockRejectedValue(Object.assign(new Error('timeout'), { code: 'ETIMEOUT' }))
+      await expect(resolveAndStoreHostnameRule(db, 'dns-rule', 'host.test')).rejects.toThrow('ETIMEOUT')
+      expect(repos.listResolvedForRule(db, 'dns-rule')).toEqual(before)
+      dns.resolve6.mockResolvedValue(['2001:4860:4860::8888'])
+      expect(await resolveAndStoreHostnameRule(db, 'dns-rule', 'host.test')).toEqual(['2001:4860:4860::8888/128', '8.8.8.8/32'])
+    } finally { sqlite.close() }
+  })
+  it('accepts authoritative per-family ENODATA but rejects NXDOMAIN and SERVFAIL', async () => {
+    dns.resolve4.mockResolvedValue(['8.8.8.8']); dns.resolve6.mockRejectedValue(Object.assign(new Error('no AAAA'), { code: 'ENODATA' }))
+    expect(await resolveHostnameToCidrs('host.test')).toEqual(['8.8.8.8/32'])
+    for (const code of ['ENOTFOUND', 'ESERVFAIL']) {
+      dns.resolve6.mockRejectedValue(Object.assign(new Error(code), { code }))
+      await expect(resolveHostnameToCidrs('host.test')).rejects.toThrow(code)
+    }
+  })
+  it('cancels both unresolved DNS queries at the shared deadline', async () => {
+    vi.useFakeTimers()
+    const pending: ((reason: Error) => void)[] = []
+    dns.resolve4.mockImplementation(() => new Promise((_, reject) => pending.push(reject)))
+    dns.resolve6.mockImplementation(() => new Promise((_, reject) => pending.push(reject)))
+    dns.cancel.mockImplementation(() => { for (const reject of pending) reject(Object.assign(new Error('cancelled'), { code: 'ECANCELLED' })) })
+    const result = expect(resolveHostnameToCidrs('host.test')).rejects.toThrow('ECANCELLED')
+    await vi.advanceTimersByTimeAsync(HOSTNAME_DNS_TIMEOUT_MS)
+    await result
+    expect(dns.cancel).toHaveBeenCalled()
+  })
+})

apps/api/src/services/source-hardening.test.ts

@@ -0,0 +1,175 @@
+import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest'
+import { createServer } from 'node:http'
+import type { AddressInfo } from 'node:net'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import * as safe from './safe-fetch.js'
+import { fetchEvobgpPages } from './evobgp.js'
+import { refreshIpList } from './lists/refresh.js'
+import { publishListCascade } from './lists/entries.js'
+
+const server = createServer((req, res) => {
+  const path = req.url!
+  if (path === '/large') { res.write('['); res.end(' '.repeat(1024) + ']'); return }
+  if (path === '/declared') { res.setHeader('Content-Length', '100000'); res.end('[]'); return }
+  if (path === '/headers') { res.setHeader('X-Large', 'x'.repeat(20_000)); res.end('{}'); return }
+  if (path === '/slow') { res.write('['); setTimeout(() => res.end(']'), 150); return }
+  if (path === '/invalid') { res.end('not JSON'); return }
+  if (path === '/compressed') { res.setHeader('Content-Encoding', 'gzip'); res.end('[]'); return }
+  if (path === '/redirect') { res.writeHead(302, { Location: '/echo' }); res.end(); return }
+  if (path === '/loop') { res.writeHead(302, { Location: '/loop' }); res.end(); return }
+  if (path === '/private') { res.writeHead(302, { Location: 'http://127.0.0.2/secret' }); res.end(); return }
+  if (path === '/cross') { res.writeHead(302, { Location: `${numericOrigin}/echo` }); res.end(); return }
+  if (path === '/cross-back') { res.writeHead(302, { Location: `${numericOrigin}/back` }); res.end(); return }
+  if (path === '/back') { res.writeHead(302, { Location: `${origin}/echo` }); res.end(); return }
+  res.setHeader('Content-Type', 'application/json')
+  res.end(JSON.stringify({ authorization: req.headers.authorization || null, cookie: req.headers.cookie || null }))
+})
+let origin: string
+let numericOrigin: string
+const databases: ReturnType<typeof createMemoryDb>[] = []
+beforeAll(async () => {
+  await new Promise<void>(resolve => server.listen(0, resolve))
+  const port = (server.address() as AddressInfo).port
+  origin = `http://localhost:${port}`
+  numericOrigin = `http://127.0.0.1:${port}`
+})
+afterEach(() => {
+  vi.restoreAllMocks(); vi.unstubAllEnvs()
+  for (const memory of databases.splice(0)) memory.sqlite.close()
+})
+afterAll(async () => { server.closeAllConnections(); await new Promise<void>(resolve => server.close(() => resolve())) })
+function allow() { vi.stubEnv('EVOFW_SOURCE_ALLOWED_ORIGINS', `${origin},${numericOrigin}`) }
+function memory() { const value = createMemoryDb(); runMigrations(value.sqlite); databases.push(value); return value }
+function feed(db: ReturnType<typeof memory>['db'], id: string, config: Record<string, unknown> = {}) {
+  repos.insertIpList(db, { id, name: id, type: 'json_url', configJson: JSON.stringify({ url: 'https://feed.test/list', ...config }) })
+  repos.replaceIpListEntries(db, id, ['8.8.8.8/32'])
+}
+
+describe('source socket boundaries', () => {
+  it('rejects private, reserved, mapped and transition IPs; accepts public v4/v6', () => {
+    for (const ip of ['127.0.0.1', '10.1.1.1', '169.254.169.254', '172.16.0.1', '192.168.1.1', '100.64.0.1', '198.18.0.1', '224.1.1.1', '::1', 'fc00::1', 'fe80::1', '::ffff:8.8.8.8', '2002:7f00:1::', '2001:db8::1']) expect(safe.isPublicSourceAddress(ip), ip).toBe(false)
+    for (const ip of ['8.8.8.8', '1.1.1.1', '2606:4700:4700::1111']) expect(safe.isPublicSourceAddress(ip), ip).toBe(true)
+  })
+  it('returns only the vetted DNS answers to the socket and rejects mixed/private resolution', async () => {
+    const resolver = vi.fn().mockResolvedValue([{ address: '8.8.8.8', family: 4 }])
+    const call = (all = false) => new Promise<unknown>((resolve, reject) => safe.sourceLookup(new URL('https://feed.test'), resolver)('feed.test', { all }, (err, address) => err ? reject(err) : resolve(address)))
+    expect(await call()).toBe('8.8.8.8')
+    expect(await call(true)).toEqual([{ address: '8.8.8.8', family: 4 }])
+    resolver.mockResolvedValue([{ address: '8.8.8.8', family: 4 }, { address: '127.0.0.1', family: 4 }])
+    await expect(call()).rejects.toThrow('not public')
+    vi.stubEnv('EVOFW_SOURCE_ALLOWED_ORIGINS', '%%%invalid')
+    await expect(call()).rejects.toThrow('policy is invalid')
+  })
+  it('blocks literal and DNS loopback without explicit origin allowlisting', async () => {
+    await expect(safe.safeFetchJson(`${numericOrigin}/echo`)).rejects.toThrow('not public')
+    await expect(safe.safeFetchJson(`${origin}/echo`)).rejects.toThrow('failed')
+    await expect(safe.safeFetchJson('file:///etc/passwd')).rejects.toThrow('HTTP(S)')
+    await expect(safe.safeFetchJson('https://user:[email protected]')).rejects.toThrow('credentials')
+  })
+  it('enforces chunked/declared byte limits, headers, JSON format, encoding and body deadline', async () => {
+    allow()
+    for (const path of ['/large', '/declared']) await expect(safe.safeFetchJson(`${origin}${path}`, { maxBytes: 32 })).rejects.toThrow('byte limit')
+    await expect(safe.safeFetchJson(`${origin}/headers`)).rejects.toThrow('failed')
+    await expect(safe.safeFetchJson(`${origin}/invalid`)).rejects.toThrow('valid JSON')
+    await expect(safe.safeFetchJson(`${origin}/compressed`)).rejects.toThrow('compressed')
+    await expect(safe.safeFetchJson(`${origin}/slow`, { timeoutMs: 30 })).rejects.toThrow(/interrupted|timed out/)
+  })
+  it('bounds redirects and strips secrets permanently on an origin change', async () => {
+    allow()
+    const headers = { Authorization: 'Bearer secret', Cookie: 'secret=value' }
+    expect((await safe.safeFetchJson(`${origin}/redirect`, { headers })).data).toEqual({ authorization: 'Bearer secret', cookie: 'secret=value' })
+    for (const path of ['/cross', '/cross-back']) expect((await safe.safeFetchJson(`${origin}${path}`, { headers })).data).toEqual({ authorization: null, cookie: null })
+    await expect(safe.safeFetchJson(`${origin}/loop`)).rejects.toThrow('redirect limit')
+    await expect(safe.safeFetchJson(`${origin}/private`, { headers })).rejects.toThrow('not public')
+  })
+})
+
+describe('bounded snapshot publication', () => {
+  it('bumps direct port-list agents and deduplicates L3 ancestor dependencies', async () => {
+    const { db, sqlite } = memory()
+    for (const id of ['port-only', 'many-parents']) repos.insertAgent(db, { id, name: id, tokenHash: id, tokenPrefix: id, status: 'approved' })
+    feed(db, 'child')
+    for (const id of ['parent-a', 'parent-b']) {
+      repos.insertIpList(db, { id, name: id, type: 'static', configJson: JSON.stringify({ items: [{ kind: 'list', value: 'child' }] }) })
+      repos.insertPolicySet(db, { id: `set-${id}`, name: id })
+      repos.insertPolicyRule(db, { id: `rule-${id}`, setId: `set-${id}`, priority: 10, action: 'deny', listId: id })
+    }
+    repos.setAgentPolicySets(db, 'many-parents', ['set-parent-a', 'set-parent-b'])
+    sqlite.prepare('INSERT INTO agent_port_rules(id,agent_id,action,protocol,port_start,port_end,src_kind,list_id) VALUES (?,?,?,?,?,?,?,?)').run('port-list', 'port-only', 'open', 'tcp', 443, 443, 'list', 'child')
+    const beforePort = repos.getAgent(db, 'port-only')!.policyGeneration
+    const beforeMany = repos.getAgent(db, 'many-parents')!.policyGeneration
+    await publishListCascade(db, 'child', { cidrs: ['1.1.1.1'] })
+    expect(repos.getAgent(db, 'port-only')!.policyGeneration).toBe(beforePort + 1)
+    expect(repos.getAgent(db, 'many-parents')!.policyGeneration).toBe(beforeMany + 1)
+    await publishListCascade(db, 'child', { cidrs: ['1.1.1.1/32'] })
+    expect(repos.getAgent(db, 'port-only')!.policyGeneration).toBe(beforePort + 1)
+    expect(repos.getAgent(db, 'many-parents')!.policyGeneration).toBe(beforeMany + 1)
+  })
+  it('quarantines unexpected empty data, preserves last success time and permits explicit empty', async () => {
+    const { db } = memory(); feed(db, 'f')
+    const before = repos.getIpList(db, 'f')!.refreshedAt
+    vi.spyOn(safe, 'safeFetchJson').mockResolvedValue({ data: [], bytes: 2 })
+    await refreshIpList(db, 'f')
+    expect(repos.listIpListEntries(db, 'f')).toHaveLength(1)
+    expect(repos.getIpList(db, 'f')!.lastError).toContain('quarantined')
+    expect(repos.getIpList(db, 'f')!.refreshedAt).toBe(before)
+    repos.updateIpList(db, 'f', { configJson: JSON.stringify({ url: 'https://feed.test', allow_empty: true }) })
+    await refreshIpList(db, 'f')
+    expect(repos.listIpListEntries(db, 'f')).toHaveLength(0)
+    expect(repos.getIpList(db, 'f')!.lastError).toBeNull()
+  })
+  it('discards stale remote results after a config edit', async () => {
+    const { db } = memory(); feed(db, 'f')
+    vi.spyOn(safe, 'safeFetchJson').mockImplementation(async () => {
+      repos.updateIpList(db, 'f', { configJson: JSON.stringify({ url: 'https://new.test' }) })
+      return { data: ['1.1.1.1'], bytes: 12 }
+    })
+    await refreshIpList(db, 'f')
+    expect(repos.getIpList(db, 'f')!.configJson).toContain('new.test')
+    expect(repos.listIpListEntries(db, 'f')[0]!.cidr).toBe('8.8.8.8/32')
+  })
+  it('coalesces the same list and limits global active refreshes to two', async () => {
+    const { db } = memory(); for (const id of ['a', 'b', 'c']) feed(db, id)
+    let active = 0; let maximum = 0
+    const fetcher = vi.spyOn(safe, 'safeFetchJson').mockImplementation(async () => {
+      active++; maximum = Math.max(active, maximum)
+      await new Promise(resolve => setTimeout(resolve, 10)); active--
+      return { data: ['1.1.1.1'], bytes: 12 }
+    })
+    const first = refreshIpList(db, 'a')
+    expect(refreshIpList(db, 'a')).toBe(first)
+    await Promise.all([first, refreshIpList(db, 'b'), refreshIpList(db, 'c')])
+    expect(maximum).toBe(2); expect(fetcher).toHaveBeenCalledTimes(3)
+  })
+  it('rejects excessive elements and incomplete/repeated cursor snapshots', async () => {
+    const fetcher = vi.spyOn(safe, 'safeFetchJson').mockResolvedValue({ data: { items: Array(safe.SOURCE_LIMITS.elements + 1).fill('x') }, bytes: 1 })
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('element limit')
+    fetcher.mockResolvedValue({ data: { items: [], has_more: true, next_cursor: 'same' }, bytes: 1 })
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('repeated')
+    fetcher.mockResolvedValue({ data: { items: [], has_more: true }, bytes: 1 })
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('Incomplete')
+    fetcher.mockResolvedValue({ data: { items: [], has_more: 'yes' }, bytes: 1 })
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('has_more')
+  })
+  it('bounds cumulative snapshot bytes, page count and elapsed pagination time', async () => {
+    let page = 0
+    const fetcher = vi.spyOn(safe, 'safeFetchJson').mockImplementation(async () => ({ data: { items: [], has_more: true, next_cursor: String(++page) }, bytes: safe.SOURCE_LIMITS.bytes }))
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('byte limit')
+    expect(fetcher).toHaveBeenCalledTimes(4)
+    page = 0; fetcher.mockClear().mockImplementation(async () => ({ data: { items: [], has_more: true, next_cursor: String(++page) }, bytes: 1 }))
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('page limit')
+    expect(fetcher).toHaveBeenCalledTimes(safe.SOURCE_LIMITS.pages)
+    let now = 1000
+    vi.spyOn(Date, 'now').mockImplementation(() => now)
+    fetcher.mockClear().mockImplementation(async () => { now += safe.SOURCE_LIMITS.timeoutMs + 1; return { data: { items: [], has_more: true, next_cursor: 'later' }, bytes: 1 } })
+    await expect(fetchEvobgpPages('https://bgp.test', 'secret', '/v1/communities', 200)).rejects.toThrow('deadline')
+    expect(fetcher).toHaveBeenCalledTimes(1)
+  })
+  it('rejects excessive JSON URL entries without replacing last good', async () => {
+    const { db } = memory(); feed(db, 'f')
+    vi.spyOn(safe, 'safeFetchJson').mockResolvedValue({ data: Array(safe.SOURCE_LIMITS.elements + 1).fill('1.1.1.1'), bytes: 1 })
+    await refreshIpList(db, 'f')
+    expect(repos.getIpList(db, 'f')!.lastError).toContain('element limit')
+    expect(repos.listIpListEntries(db, 'f')[0]!.cidr).toBe('8.8.8.8/32')
+  })
+})

apps/api/src/services/telemetry.test.ts

@@ -0,0 +1,196 @@
+import { afterEach, describe, expect, it } from 'vitest'
+import { createMemoryDb, runMigrations, repos } from '@evofw/db'
+import { applyReportBodySchema, type ApplyReportV2Body } from '@evofw/shared'
+import { ingestTelemetry } from './telemetry.js'
+import { mapAgentDiagnostics } from './row-mappers.js'
+
+const opened: ReturnType<typeof createMemoryDb>[] = []
+const BASE_TIME = Date.now() - 86_400_000
+afterEach(() => { for (const mem of opened.splice(0)) mem.sqlite.close() })
+function memory() {
+  const mem = createMemoryDb(); opened.push(mem); runMigrations(mem.sqlite)
+  repos.insertAgent(mem.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: 'a', status: 'approved' })
+  return mem
+}
+function report(sequence: number, packets = 100, extra: Partial<ApplyReportV2Body> = {}): ApplyReportV2Body {
+  return {
+    telemetry_version: 2, report_id: `r-${sequence}`, sequence, boot_id: 'boot', observed_at: new Date(BASE_TIME + sequence * 60_000).toISOString(),
+    interval_ms: 60_000, counter_epoch: { traffic: 't', ip: 'i', port: 'p' }, capabilities: ['policy_ipv4'],
+    quality: { partial: false, truncated: false, unsupported: [], lost_reports: 0 }, status: 'ok', packets_dropped: packets, packets_accepted: packets,
+    ip_hits: [{ ip: '8.8.8.8', packets }], port_hits: [{ ip: '8.8.8.8', port: 22, protocol: 'tcp', packets }],
+    ...extra,
+  }
+}
+const ingest = (db: ReturnType<typeof memory>['db'], body: ApplyReportV2Body) => ingestTelemetry(db, 'a', body, '127.0.0.1')
+
+describe('telemetry v2 integrity', () => {
+  it('classifies an explicit unsupported-capability apply error before generic failed status', () => {
+    const { db } = memory()
+    const body = applyReportBodySchema.parse(report(1, 100, { status: 'error', apply_error: { code: 'UNSUPPORTED_CAPABILITY', message: 'IPv6 backend unavailable' } }))
+    ingestTelemetry(db, 'a', body, '127.0.0.1')
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('unsupported')
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).apply_error!.code).toBe('UNSUPPORTED_CAPABILITY')
+  })
+
+  it('recovers from status:error without optional text when the same acknowledgement succeeds', () => {
+    const { db } = memory()
+    const hash = `sha256:${'a'.repeat(64)}`
+    repos.updateAgent(db, 'a', { desiredHash: hash, desiredHashGeneration: 1 })
+    for (const [sequence, status] of [[1, 'ok'], [2, 'error'], [3, 'ok']] as const) {
+      const body = applyReportBodySchema.parse(report(sequence, 100 + sequence, { status, applied_hash: hash, applied_revision: 1 }))
+      ingestTelemetry(db, 'a', body, '127.0.0.1')
+      expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe(status === 'error' ? 'failed' : 'in_sync')
+    }
+  })
+
+  it('marks detail baseline/decrease intervals partial even when traffic and producer quality are complete', () => {
+    const { db } = memory()
+    for (const [sequence, detail] of [[1, 100], [2, 105], [3, 1]] as const) {
+      const body = applyReportBodySchema.parse(report(sequence, 100 + sequence, {
+        ip_hits: [{ ip: '8.8.8.8', packets: detail }], port_hits: [{ ip: '8.8.8.8', port: 22, protocol: 'tcp', packets: detail }],
+        detail_counts: { ip_found: 1, ip_sent: 1, port_found: 1, port_sent: 1 },
+      }))
+      ingestTelemetry(db, 'a', body, '127.0.0.1')
+      expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).quality.partial).toBe(sequence !== 2)
+    }
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(5)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(5)
+  })
+
+  it('requires the complete v2 envelope and rejects unsafe or negative counters', () => {
+    expect(applyReportBodySchema.safeParse({ telemetry_version: 2, status: 'ok' }).success).toBe(false)
+    for (const packets of [-1, Number.MAX_SAFE_INTEGER + 1, 1.5]) expect(applyReportBodySchema.safeParse(report(1, packets)).success).toBe(false)
+    expect(applyReportBodySchema.safeParse(report(1)).success).toBe(true)
+    expect(applyReportBodySchema.safeParse({ status: 'ok' }).success).toBe(true)
+    expect(applyReportBodySchema.safeParse(report(1, 100, { detail_counts: { ip_found: 1, ip_sent: 2, port_found: 1, port_sent: 1 } })).success).toBe(false)
+    expect(applyReportBodySchema.safeParse(report(1, 100, { ip_hits: [{ ip: '::1', packets: 1 }, { ip: '0:0:0:0:0:0:0:1', packets: 2 }] })).success).toBe(false)
+  })
+
+  it('persists detail found/sent coverage, marks truncation and avoids unsafe lifetime rounding', () => {
+    const { db } = memory()
+    ingest(db, report(1))
+    repos.updateAgent(db, 'a', { totalPacketsDropped: Number.MAX_SAFE_INTEGER })
+    ingest(db, report(2, 105, { detail_counts: { ip_found: 50, ip_sent: 1, port_found: 10, port_sent: 1 } }))
+    const diagnostics = mapAgentDiagnostics(repos.getAgent(db, 'a')!)
+    expect(diagnostics.detail_counts).toEqual({ ip_found: 50, ip_sent: 1, port_found: 10, port_sent: 1 })
+    expect(diagnostics.quality.truncated).toBe(true)
+    expect(diagnostics.quality.unsupported).toContain('traffic_totals_overflow')
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(Number.MAX_SAFE_INTEGER)
+    expect(repos.getTelemetryState(db, 'a')!.droppedBaseline).toBe(105)
+  })
+
+  it('deduplicates reports and rejects delayed/out-of-order updates before every effect', () => {
+    const { db } = memory()
+    expect(ingest(db, report(1)).accepted).toBe(true)
+    expect(ingest(db, report(2, 105)).accepted).toBe(true)
+    const before = repos.getAgent(db, 'a')
+    expect(ingest(db, report(2, 999)).duplicate).toBe(true)
+    expect(ingest(db, report(1, 999, { report_id: 'different-old-id' })).accepted).toBe(false)
+    expect(ingest(db, report(3, 999, { report_id: 'r-1' })).duplicate).toBe(true)
+    expect(repos.getAgent(db, 'a')).toEqual(before)
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(2)
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(5)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(5)
+    expect(ingestTelemetry(db, 'a', { status: 'ok', packets_dropped: 999 }, '127.0.0.1').accepted).toBe(false)
+  })
+
+  it('UI reset preserves baselines and samples: 100 -> reset -> 100 -> 105 adds 0,0,5', () => {
+    const { db } = memory()
+    ingest(db, report(1, 80)); ingest(db, report(2, 100))
+    repos.resetDisplayedStats(db, 'a')
+    expect(repos.getAgent(db, 'a')!.lastApplyPacketsDropped).toBe(100)
+    expect(repos.listStatsSamples(db, 'a')).toHaveLength(2)
+    ingest(db, report(3, 100))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(0)
+    expect(repos.listIpBlockStats(db, 'a')).toEqual([])
+    expect(repos.listPortBlockStatsAggregate(db, 'a')).toEqual([])
+    ingest(db, report(4, 105))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(5)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(5)
+  })
+
+  it('independent traffic/IP/port epochs establish baselines even when new absolutes are larger', () => {
+    const { db } = memory()
+    ingest(db, report(1)); ingest(db, report(2, 105))
+    ingest(db, report(3, 1000, { counter_epoch: { traffic: 'new', ip: 'i', port: 'p' } }))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(900)
+    ingest(db, report(4, 1005, { counter_epoch: { traffic: 'new', ip: 'new-i', port: 'p' } }))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(10)
+    expect(repos.listIpBlockStats(db, 'a')[0]!.packets).toBe(900)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(905)
+    ingest(db, report(5, 2000, { boot_id: 'reboot', counter_epoch: { traffic: 'new', ip: 'new-i', port: 'p' } }))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(10)
+    expect(repos.listPortBlockStatsAggregate(db, 'a')[0]!.packets).toBe(905)
+  })
+
+  it('missing counter values and omitted detail rows preserve baselines; decreasing counters are partial', () => {
+    const { db } = memory()
+    ingest(db, report(1))
+    expect(repos.listStatsSamples(db, 'a')[0]!.deltaDropped).toBeNull()
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).quality.partial).toBe(true)
+    ingest(db, report(2, 100, { packets_dropped: undefined, packets_accepted: undefined, ip_hits: [], port_hits: [] }))
+    expect(repos.listStatsSamples(db, 'a').find((sample) => sample.sequence === 2)!.deltaDropped).toBeNull()
+    expect(repos.getTelemetryState(db, 'a')!.droppedBaseline).toBe(100)
+    ingest(db, report(3, 105))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+    ingest(db, report(4, 1))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).quality.partial).toBe(true)
+  })
+
+  it('flags increasing sequence with a backward observation clock and leaves interval coverage unknown', () => {
+    const { db } = memory()
+    ingest(db, report(1))
+    ingest(db, report(2, 105, { observed_at: '2020-01-01T00:00:00Z' }))
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+    const sample = repos.listStatsSamples(db, 'a').find((row) => row.sequence === 2)!
+    expect(sample.intervalMs).toBeNull()
+    expect(JSON.parse(sample.qualityJson!).unsupported).toContain('observed_at_out_of_order')
+    expect(repos.getTelemetryState(db, 'a')!.observedAt).toBe(report(1).observed_at)
+  })
+
+  it('rolls back baseline, totals, details, diagnostics and replay watermark if sample persistence fails', () => {
+    const { db, sqlite } = memory()
+    ingest(db, report(1))
+    const before = repos.getAgent(db, 'a')
+    sqlite.exec("CREATE TRIGGER fail_sample BEFORE INSERT ON agent_stats_samples BEGIN SELECT RAISE(ABORT, 'injected sample failure'); END;")
+    expect(() => ingest(db, report(2, 105))).toThrow('injected sample failure')
+    expect(repos.getAgent(db, 'a')).toEqual(before)
+    expect(repos.getTelemetryState(db, 'a')!.sequence).toBe(1)
+    expect(repos.listIpBlockStats(db, 'a')).toEqual([])
+    sqlite.exec('DROP TRIGGER fail_sample')
+    expect(ingest(db, report(2, 105)).accepted).toBe(true)
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(5)
+  })
+
+  it('keeps sequence watermark across raw retention and marks legacy telemetry limited', () => {
+    const { db } = memory()
+    ingestTelemetry(db, 'a', { status: 'ok', packets_dropped: 100 }, '127.0.0.1')
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('unsupported')
+    ingest(db, report(5, 1000))
+    repos.deleteStatsSamplesForAgent(db, 'a')
+    expect(ingest(db, report(3, 999, { report_id: 'old' })).accepted).toBe(false)
+    expect(repos.getAgent(db, 'a')!.totalPacketsDropped).toBe(100)
+  })
+
+  it('diagnostics distinguish acknowledged content, drift, apply failures and later successful retries', () => {
+    const { db } = memory()
+    const desired = `sha256:${'a'.repeat(64)}`
+    repos.updateAgent(db, 'a', { desiredHash: desired, desiredHashGeneration: 1, lastPolicyDeliveredAt: '2026-01-01T00:00:00Z' })
+    ingest(db, report(1, 100, { applied_hash: desired, applied_revision: 1 }))
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('in_sync')
+    ingest(db, report(2, 101, { status: 'error', apply_error: { code: 'KERNEL_CHECK_FAILED', message: 'candidate invalid' } }))
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('failed')
+    ingest(db, report(3, 102))
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('failed')
+    ingest(db, report(4, 103, { applied_hash: desired, applied_revision: 1 }))
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('in_sync')
+    ingest(db, report(5, 104, { applied_hash: `sha256:${'b'.repeat(64)}`, applied_revision: 1 }))
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('drift')
+    ingest(db, report(6, 105, { applied_hash: desired, applied_revision: 1 }))
+    expect(mapAgentDiagnostics(repos.getAgent(db, 'a')!).status).toBe('in_sync')
+  })
+})

apps/api/src/services/telemetry.ts

@@ -0,0 +1,158 @@
+import { createHash } from 'node:crypto'
+import { repos, type Db } from '@evofw/db'
+import { type ApplyReportBody, type TelemetryQuality, hostFwRuleSchema, hostListenerSchema } from '@evofw/shared'
+import { buildStatsRollups } from './analytics.js'
+
+const limitedQuality: TelemetryQuality = { partial: true, truncated: false, unsupported: ['telemetry_v2'], lost_reports: 0 }
+const epochKey = (boot: string, epoch: string) => JSON.stringify([boot, epoch])
+
+/** All report effects share one transaction; replay decisions precede every write. */
+export function ingestTelemetry(db: Db, agentId: string, body: ApplyReportBody, contactIp: string, now = new Date().toISOString()) {
+  return db.transaction((tx) => {
+    const agent = repos.getAgent(tx, agentId)
+    if (!agent) throw new Error('Agent not found')
+    const state = repos.getTelemetryState(tx, agentId)
+    const v2 = body.telemetry_version === 2 ? body : null
+    if (v2 && (repos.hasTelemetryReport(tx, agentId, v2.report_id) || state?.reportId === v2.report_id)) return { ok: true, accepted: false, duplicate: true }
+    if (v2 && state && v2.sequence <= state.sequence) return { ok: true, accepted: false, duplicate: false, reason: 'stale_sequence' }
+    // Delayed legacy payloads cannot overwrite newer v2 baselines or diagnostics.
+    if (!v2 && state) return { ok: true, accepted: false, duplicate: false, reason: 'legacy_after_v2' }
+
+    const quality: TelemetryQuality = v2 ? { ...v2.quality, unsupported: [...v2.quality.unsupported] } : { ...limitedQuality, unsupported: [...limitedQuality.unsupported] }
+    const markPartial = (feature: string) => {
+      quality.partial = true
+      if (!quality.unsupported.includes(feature)) quality.unsupported.push(feature)
+    }
+    if (v2 && state) quality.lost_reports = Math.max(quality.lost_reports, v2.sequence - state.sequence - 1)
+    if (quality.lost_reports > 0) quality.partial = true
+    const hasDetailCollection = !!(v2 && (v2.ip_hits !== undefined || v2.port_hits !== undefined || v2.capabilities.some((feature) => /^(ip|port)[-_]counters/.test(feature))))
+    if (v2 && hasDetailCollection && !v2.detail_counts) markPartial('detail_counts')
+    if (v2?.detail_counts && (v2.detail_counts.ip_sent < v2.detail_counts.ip_found || v2.detail_counts.port_sent < v2.detail_counts.port_found)) {
+      quality.truncated = true
+      quality.partial = true
+    }
+    const observationOutOfOrder = !!(v2 && state && Date.parse(v2.observed_at) < Date.parse(state.observedAt))
+    if (observationOutOfOrder) markPartial('observed_at_out_of_order')
+    const observationFuture = !!(v2 && Date.parse(v2.observed_at) > Date.parse(now) + 5 * 60_000)
+    if (observationFuture) markPartial('observed_at_future')
+    const trafficEpoch = v2 ? epochKey(v2.boot_id, v2.counter_epoch.traffic) : null
+    const sameEpoch = !!(v2 && state && trafficEpoch === state.trafficEpoch)
+    const nextCounter = (incoming: number | undefined, previous: number | null | undefined, feature: string) => {
+      if (incoming === undefined) return { baseline: sameEpoch ? previous ?? null : null, delta: null }
+      if (!v2) return { baseline: incoming, delta: incoming >= (previous ?? 0) ? incoming - (previous ?? 0) : incoming }
+      if (!sameEpoch || previous === null || previous === undefined) return { baseline: incoming, delta: null }
+      if (incoming < previous) { markPartial(`${feature}_counter_decrease`); return { baseline: incoming, delta: null } }
+      return { baseline: incoming, delta: incoming - previous }
+    }
+    const dropped = nextCounter(body.packets_dropped, v2 ? state?.droppedBaseline : agent.lastApplyPacketsDropped, 'dropped')
+    const accepted = nextCounter(body.packets_accepted, v2 ? state?.acceptedBaseline : agent.lastApplyPacketsAccepted, 'accepted')
+    if (body.packets_dropped === undefined || body.packets_accepted === undefined) markPartial('traffic_counters')
+    if (dropped.delta === null || accepted.delta === null) quality.partial = true
+    const addTotal = (total: number, delta: number) => {
+      const result = total + delta
+      if (!Number.isSafeInteger(result)) { markPartial('traffic_totals_overflow'); return total }
+      return result
+    }
+    const totalDropped = addTotal(agent.totalPacketsDropped, dropped.delta ?? 0)
+    const totalAccepted = addTotal(agent.totalPacketsAccepted, accepted.delta ?? 0)
+    const observed = v2?.observed_at ?? now
+    const validStart = (at: string | null | undefined, delta: number | null) => !observationOutOfOrder && !observationFuture && delta !== null && at && Date.parse(at) < Date.parse(observed) ? at : null
+    const droppedStart = validStart(v2 ? sameEpoch ? state?.droppedBaselineAt : null : agent.lastTelemetryAt, dropped.delta)
+    const acceptedStart = validStart(v2 ? sameEpoch ? state?.acceptedBaselineAt : null : agent.lastTelemetryAt, accepted.delta)
+    if (!droppedStart || !acceptedStart) quality.partial = true
+    const baselineAt = (incoming: number | undefined, previous: string | null | undefined) => incoming === undefined
+      ? sameEpoch ? previous ?? null : null
+      : observationOutOfOrder || observationFuture ? null : observed
+
+    if (!v2 && body.packets_dropped !== undefined && body.packets_dropped < agent.lastApplyPacketsDropped) repos.resetIpBlockStatsBaselines(tx, agentId)
+    if (body.ip_hits?.length) {
+      if (v2 && !v2.counter_epoch.ip && body.source !== 'mikrotik') markPartial('ip_counter_epoch')
+      else repos.upsertIpBlockStats(tx, agentId, body.ip_hits, now, {
+        mode: body.source === 'mikrotik' ? 'presence' : 'absolute',
+        ...(v2?.counter_epoch.ip ? { counterEpoch: epochKey(v2.boot_id, v2.counter_epoch.ip) } : {}),
+        onOverflow: () => markPartial('ip_totals_overflow'),
+        onUnknownInterval: () => { quality.partial = true },
+      })
+    }
+    if (body.port_hits?.length && body.source !== 'mikrotik') {
+      if (v2 && !v2.counter_epoch.port) markPartial('port_counter_epoch')
+      else repos.upsertPortBlockStats(tx, agentId, body.port_hits, now, {
+        ...(v2?.counter_epoch.port ? { counterEpoch: epochKey(v2.boot_id, v2.counter_epoch.port) } : {}),
+        onOverflow: () => markPartial('port_totals_overflow'),
+        onUnknownInterval: () => { quality.partial = true },
+      })
+    }
+    const evicted = repos.enforceDetailBudget(tx, agentId)
+    if (evicted.ip || evicted.port) { markPartial('detail_storage_budget'); quality.truncated = true }
+
+    const succeeded = body.status === 'ok'
+    const reportedError = body.apply_error ?? (body.error ? { code: 'AGENT_APPLY_ERROR', message: body.error } : null)
+    const acknowledged = !!(v2 && succeeded && v2.applied_hash)
+    const newAcknowledgement = !!(v2 && acknowledged && (v2.applied_hash !== agent.appliedHash || (v2.applied_revision ?? null) !== agent.appliedRevision || agent.lastApplyError || (agent.lastApplyStatus !== null && agent.lastApplyStatus !== 'ok')))
+    const clearApplyError = !v2 || acknowledged
+    repos.updateAgent(tx, agentId, {
+      telemetryVersion: v2 ? 2 : 1,
+      lastTelemetryAt: now,
+      lastApplyAt: !v2 || newAcknowledgement || !succeeded ? now : agent.lastApplyAt,
+      lastApplyStatus: !v2 || !succeeded || newAcknowledgement ? body.status : agent.lastApplyStatus,
+      lastApplyError: reportedError?.message ?? (clearApplyError ? null : agent.lastApplyError),
+      applyErrorJson: reportedError ? JSON.stringify(reportedError) : clearApplyError ? null : agent.applyErrorJson,
+      lastApplyPrefixCount: body.prefix_count ?? agent.lastApplyPrefixCount,
+      lastApplyPacketsDropped: body.packets_dropped ?? agent.lastApplyPacketsDropped,
+      lastApplyPacketsAccepted: body.packets_accepted ?? agent.lastApplyPacketsAccepted,
+      totalPacketsDropped: totalDropped,
+      totalPacketsAccepted: totalAccepted,
+      lastApplyKernelMethod: body.kernel_method ?? agent.lastApplyKernelMethod,
+      lastSeenAt: now,
+      lastSeenIp: contactIp,
+      ...(v2 ? {
+        clientVersion: v2.agent_version ?? agent.clientVersion,
+        capabilitiesJson: JSON.stringify(v2.capabilities),
+        appliedHash: succeeded ? v2.applied_hash ?? agent.appliedHash : agent.appliedHash,
+        appliedRevision: acknowledged ? v2.applied_revision ?? null : agent.appliedRevision,
+      } : {}),
+      qualityJson: JSON.stringify(quality),
+      detailCountsJson: v2?.detail_counts ? JSON.stringify(v2.detail_counts) : null,
+    })
+    repos.insertStatsSample(tx, {
+      id: crypto.randomUUID(), agentId,
+      packetsDropped: body.packets_dropped ?? agent.lastApplyPacketsDropped,
+      packetsAccepted: body.packets_accepted ?? agent.lastApplyPacketsAccepted,
+      prefixCount: body.prefix_count ?? agent.lastApplyPrefixCount ?? 0,
+      kernelMethod: body.kernel_method ?? agent.lastApplyKernelMethod,
+      recordedAt: now,
+      telemetryVersion: v2 ? 2 : 1,
+      reportId: v2?.report_id, sequence: v2?.sequence,
+      observedAt: v2?.observed_at ?? now,
+      intervalMs: droppedStart && acceptedStart ? Date.parse(observed) - Math.min(Date.parse(droppedStart), Date.parse(acceptedStart)) : null,
+      droppedIntervalStart: droppedStart, acceptedIntervalStart: acceptedStart,
+      droppedPresent: body.packets_dropped === undefined ? 0 : 1,
+      acceptedPresent: body.packets_accepted === undefined ? 0 : 1,
+      analyticsProcessed: 1,
+      deltaDropped: dropped.delta, deltaAccepted: accepted.delta,
+      qualityJson: JSON.stringify(quality),
+      detailCountsJson: v2?.detail_counts ? JSON.stringify(v2.detail_counts) : null,
+    })
+    if (v2) repos.upsertTelemetryState(tx, {
+      agentId, sequence: v2.sequence, reportId: v2.report_id,
+      bootId: v2.boot_id, trafficEpoch: trafficEpoch!, observedAt: observationOutOfOrder || observationFuture ? state?.observedAt ?? now : v2.observed_at, receivedAt: now,
+      droppedBaseline: dropped.baseline, acceptedBaseline: accepted.baseline,
+      droppedBaselineAt: baselineAt(body.packets_dropped, state?.droppedBaselineAt),
+      acceptedBaselineAt: baselineAt(body.packets_accepted, state?.acceptedBaselineAt),
+    })
+    repos.addStatsRollups(tx, buildStatsRollups(agentId,
+      { start: droppedStart, end: observed, delta: dropped.delta },
+      { start: acceptedStart, end: observed, delta: accepted.delta }, quality, now, !v2))
+
+    if (body.host_firewall && body.source !== 'mikrotik') {
+      const rules = body.host_firewall.rules.flatMap((item) => { const parsed = hostFwRuleSchema.safeParse(item); return parsed.success ? [parsed.data] : [] })
+      const listeners = body.host_firewall.listeners.flatMap((item) => { const parsed = hostListenerSchema.safeParse(item); return parsed.success ? [parsed.data] : [] })
+      if (rules.length || listeners.length || !repos.getHostFirewallSnapshot(tx, agentId)) {
+        const payload = JSON.stringify({ rules, listeners })
+        const digest = createHash('sha256').update(payload).digest('hex').slice(0, 16)
+        repos.upsertHostFirewallSnapshot(tx, agentId, payload, now, digest)
+      }
+    }
+    return { ok: true, accepted: true, duplicate: false }
+  })
+}

apps/api/src/services/ui-summary.test.ts

@@ -0,0 +1,135 @@
+import { afterEach, describe, expect, it } from 'vitest'
+import { buildApp } from '../app.js'
+import { loadConfig } from '../config.js'
+import { repos } from '@evofw/db'
+import { uiSummarySchema } from '@evofw/shared'
+import { hashToken } from '../plugins/auth.js'
+
+const apps: Awaited<ReturnType<typeof buildApp>>[] = []
+afterEach(async () => { for (const app of apps.splice(0)) await app.close() })
+async function fixture() {
+  const config = { ...loadConfig(), authRequired: true, jwtSecret: 'test-secret-key', authIssuer: 'https://auth.test', staticDir: null, logLevel: 'error' }
+  const app = await buildApp({ memory: true, config }); apps.push(app); await app.ready()
+  repos.insertAgent(app.db, { id: 'a', name: 'a', tokenPrefix: 'a', tokenHash: hashToken('token-a'), status: 'approved', lastSeenAt: new Date().toISOString(), totalPacketsDropped: 15 })
+  repos.insertAgent(app.db, { id: 'b', name: 'b', tokenPrefix: 'b', tokenHash: 'token-b', status: 'approved' })
+  repos.insertAgent(app.db, { id: 'c', name: 'c', tokenPrefix: 'c', tokenHash: 'token-c', status: 'pending' })
+  repos.insertIpList(app.db, { id: 'l', name: 'l', type: 'static', lastError: 'upstream failed' })
+  const headers = (permissions: string[], allowedApps = ['fw']) => ({ authorization: `Bearer ${app.jwt.sign({ sub: 'viewer', apps: allowedApps, permissions, iss: config.authIssuer })}` })
+  return { app, headers }
+}
+
+describe('UI summary and telemetry routes', () => {
+  it('exposes explain under agents permission and records bounded override TTL/reason with server-owned author', async () => {
+    const { app, headers } = await fixture()
+    const write = headers(['fw:agents:write']), read = headers(['fw:agents:read'])
+    const create = await app.inject({ method: 'POST', url: '/api/v1/agents/a/overrides', headers: write, payload: { cidr: '8.8.8.8', action: 'deny', reason: 'Investigation', ttl_seconds: 60, created_by_user_id: 'spoofed' } })
+    expect(create.statusCode).toBe(200)
+    expect(create.json()).toMatchObject({ cidr: '8.8.8.8/32', created_by_user_id: 'viewer', reason: 'Investigation', expired: false, expiration_enforcement: 'next_policy_sync' })
+    expect(Date.parse(create.json().expires_at) - Date.now()).toBeGreaterThan(55_000)
+    const explain = await app.inject({ url: '/api/v1/agents/a/explain?ip=8.8.8.8', headers: read })
+    expect(explain.statusCode).toBe(200)
+    expect(explain.json().desired).toMatchObject({ decision: 'drop', stage: 'deny', match: { override_id: create.json().id } })
+    expect((await app.inject({ url: '/api/v1/agents/a/explain?ip=8.8.8.8', headers: headers(['fw:stats:read']) })).statusCode).toBe(403)
+    const both = await app.inject({ method: 'POST', url: '/api/v1/agents/a/overrides', headers: write, payload: { cidr: '1.1.1.1', action: 'deny', ttl_seconds: 60, expires_at: new Date(Date.now() + 60_000).toISOString() } })
+    expect(both.statusCode).toBe(400)
+    const duplicate = await app.inject({ method: 'POST', url: '/api/v1/agents/a/overrides', headers: write, payload: { cidr: '8.8.8.8', action: 'allow' } })
+    expect(duplicate.statusCode).toBe(409)
+    app.sqlite.prepare('UPDATE ip_overrides SET expires_at=? WHERE id=?').run(new Date(Date.now() - 1000).toISOString(), create.json().id)
+    const before = repos.getAgent(app.db, 'a')!.policyGeneration
+    const policy = await app.inject({ url: '/v1/agent/policy', headers: { authorization: 'Bearer token-a' } })
+    expect(policy.statusCode).toBe(200)
+    expect(policy.json().deny_cidrs).not.toContain('8.8.8.8/32')
+    expect(policy.json().generation).toBe(before + 1)
+    expect((await app.inject({ url: '/api/v1/agents/a/overrides', headers: read })).json().items[0].expired).toBe(true)
+    const replacement = await app.inject({ method: 'POST', url: '/api/v1/agents/a/overrides', headers: write, payload: { cidr: '8.8.8.8', action: 'deny', reason: 'New investigation', ttl_seconds: 60 } })
+    expect(replacement.statusCode).toBe(200)
+  })
+
+  it('accepts a real >1MiB 100k-prefix import with a scoped body limit and rejects excess elements', async () => {
+    const { app, headers } = await fixture()
+    const entries = Array.from({ length: 100_000 }, (_, i) => `10.${i >> 16}.${(i >> 8) & 255}.${i & 255}`)
+    const payload = { name: 'large-manual', type: 'static', entries }
+    expect(Buffer.byteLength(JSON.stringify(payload))).toBeGreaterThan(1024 * 1024)
+    const result = await app.inject({ method: 'POST', url: '/api/v1/lists', headers: headers(['fw:lists:write']), payload })
+    expect(result.statusCode).toBe(200)
+    expect(repos.countEntriesByListIds(app.db, [result.json().id]).get(result.json().id)).toBe(100_000)
+    const excessive = await app.inject({ method: 'POST', url: `/api/v1/lists/${result.json().id}/entries`, headers: headers(['fw:lists:write']), payload: { values: [Array(100_001).fill('8.8.8.8').join(' ')] } })
+    expect(excessive.statusCode).toBe(400)
+    const unrelated = await app.inject({ method: 'PATCH', url: '/api/v1/agents/a', headers: headers(['fw:agents:write']), payload: { name: 'x'.repeat(1024 * 1024 + 1) } })
+    expect(unrelated.statusCode).toBe(413)
+  }, 15_000)
+
+  it('protects bounded analytics/CSV/raw endpoints with stats permission and paginates tied raw timestamps without loss', async () => {
+    const { app, headers } = await fixture()
+    const from = '2026-01-01T00:00:00.000Z', to = '2026-01-01T01:00:00.000Z'
+    const query = `from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}&resolution=1m&agent_ids=a`
+    expect((await app.inject({ url: `/api/v1/stats/timeseries?${query}`, headers: headers(['fw:agents:read']) })).statusCode).toBe(403)
+    const statsHeaders = headers(['fw:stats:read'])
+    const response = await app.inject({ url: `/api/v1/stats/timeseries?${query}`, headers: statsHeaders })
+    expect(response.statusCode).toBe(200)
+    expect(response.json().items).toHaveLength(60)
+    expect(response.json().items.every((item: { packets_dropped: unknown }) => item.packets_dropped === null)).toBe(true)
+    const csv = await app.inject({ url: `/api/v1/stats/timeseries.csv?${query}`, headers: statsHeaders })
+    expect(csv.statusCode).toBe(200)
+    expect(csv.headers['content-type']).toContain('text/csv')
+    expect(csv.body.split('\r\n')).toHaveLength(62)
+    for (const id of ['raw-a', 'raw-b', 'raw-c']) repos.insertStatsSample(app.db, { id, agentId: 'a', recordedAt: from, packetsDropped: 100 })
+    const path = `/api/v1/agents/a/stats/raw?from=${encodeURIComponent(from)}&to=${encodeURIComponent(to)}&limit=2`
+    const page1 = await app.inject({ url: path, headers: statsHeaders })
+    expect(page1.statusCode).toBe(200)
+    expect(page1.json().items.map((item: { id: string }) => item.id)).toEqual(['raw-a', 'raw-b'])
+    expect(page1.json().has_more).toBe(true)
+    const page2 = await app.inject({ url: `${path}&cursor=${page1.json().next_cursor}`, headers: statsHeaders })
+    expect(page2.json().items.map((item: { id: string }) => item.id)).toEqual(['raw-c'])
+    expect(page2.json().next_cursor).toBeNull()
+    expect((await app.inject({ url: `${path}&cursor=invalid`, headers: statsHeaders })).statusCode).toBe(400)
+    expect((await app.inject({ url: `${path.replace('limit=2', 'limit=501')}`, headers: statsHeaders })).statusCode).toBe(400)
+  })
+
+  it('requires authentication/fw membership and independently omits unauthorized sections', async () => {
+    const { app, headers } = await fixture()
+    expect((await app.inject({ url: '/api/v1/ui/summary' })).statusCode).toBe(401)
+    expect((await app.inject({ url: '/api/v1/ui/summary', headers: headers([], []) })).statusCode).toBe(403)
+    const empty = await app.inject({ url: '/api/v1/ui/summary', headers: headers([]) })
+    expect(empty.statusCode).toBe(200)
+    expect(Object.keys(empty.json())).toEqual(['updated_at'])
+    for (const [scope, field] of [['agents', 'agents'], ['lists', 'lists'], ['policies', 'policies'], ['dashboard', 'dashboard']] as const) {
+      const response = await app.inject({ url: '/api/v1/ui/summary', headers: headers([`fw:${scope}:read`]) })
+      expect(response.statusCode).toBe(200)
+      expect(Object.keys(response.json()).sort()).toEqual([field, 'updated_at'].sort())
+      expect(uiSummarySchema.safeParse(response.json()).success).toBe(true)
+    }
+    const all = await app.inject({ url: '/api/v1/ui/summary', headers: headers(['fw:agents:write', 'fw:lists:write', 'fw:policies:write', 'fw:dashboard:read']) })
+    expect(all.json().agents).toEqual({ approved: 2, online: 1, pending: 1, offline: 1 })
+    expect(all.json().lists).toEqual({ total: 1, failed: 1 })
+    expect(all.json().dashboard.packets_dropped).toBe(15)
+    expect(all.body).not.toContain('token-a')
+    expect(all.body).not.toContain('upstream failed')
+  })
+
+  it('acknowledges a committed v2 report ID and preserves history/baselines through stats reset', async () => {
+    const { app, headers } = await fixture()
+    const post = async (sequence: number, value: number, reportId = `r-${sequence}`) => app.inject({
+      method: 'POST', url: '/v1/agent/apply-report', headers: { authorization: 'Bearer token-a' },
+      payload: { telemetry_version: 2, report_id: reportId, sequence, boot_id: 'boot', observed_at: new Date().toISOString(), interval_ms: 60_000,
+        counter_epoch: { traffic: 'e' }, capabilities: ['policy_ipv4'], quality: { partial: false, truncated: false, unsupported: [], lost_reports: 0 }, status: 'ok', packets_dropped: value, packets_accepted: 0 },
+    })
+    const first = await post(1, 100)
+    expect(first.statusCode).toBe(200)
+    expect(first.json()).toMatchObject({ ok: true, accepted: true, report_id: 'r-1', sequence: 1 })
+    const replay = await post(1, 100)
+    expect(replay.json()).toMatchObject({ ok: true, duplicate: true, report_id: 'r-1' })
+    const reset = await app.inject({ method: 'POST', url: '/api/v1/agents/a/stats/reset', headers: headers(['fw:agents:write']) })
+    expect(reset.statusCode).toBe(200)
+    await post(2, 100)
+    expect(repos.getAgent(app.db, 'a')!.totalPacketsDropped).toBe(0)
+    await post(3, 105)
+    expect(repos.getAgent(app.db, 'a')!.totalPacketsDropped).toBe(5)
+    expect(repos.listStatsSamples(app.db, 'a')).toHaveLength(3)
+    const stale = await post(2, 999, 'restored-flash')
+    expect(stale.json()).toMatchObject({ ok: true, accepted: false, reason: 'stale_sequence', report_id: 'restored-flash', sequence: 2, sequence_floor: 3 })
+    expect(repos.getAgent(app.db, 'a')!.totalPacketsDropped).toBe(5)
+    app.sqlite.prepare('UPDATE agent_telemetry_state SET sequence=? WHERE agent_id=?').run(Number.MAX_SAFE_INTEGER, 'a')
+    expect((await post(1, 999, 'exhausted')).json().sequence_floor).toBe(Number.MAX_SAFE_INTEGER)
+  })
+})

apps/api/src/services/uniq.ts

@@ -1,6 +1,8 @@
-/** Trim, drop empty, dedupe (order not guaranteed — sorted for stable hashes). */
+import { canonicalCidr } from '@evofw/shared'
+
+/** Validate, canonicalize and deduplicate networks for stable policy hashes. */
 export function uniqCidrs(cidrs: readonly string[]): string[] {
-  return [...new Set(cidrs.map((c) => c.trim()).filter(Boolean))].sort()
+  return [...new Set(cidrs.map(canonicalCidr))].sort()
 }
 
 /** Trim, drop empty, dedupe preserving first-seen order. */
@@ -8,8 +10,8 @@ export function uniqCidrsPreserveOrder(cidrs: readonly string[]): string[] {
   const seen = new Set<string>()
   const out: string[] = []
   for (const c of cidrs) {
-    const t = c.trim()
+    const t = canonicalCidr(c)
-    if (!t || seen.has(t)) continue
+    if (seen.has(t)) continue
     seen.add(t)
     out.push(t)
   }

apps/api/src/services/webhook-transport.test.ts

@@ -0,0 +1,52 @@
+import { afterEach, describe, expect, it, vi } from 'vitest'
+import { EventEmitter } from 'node:events'
+const dns = vi.hoisted(() => ({ lookup: vi.fn() }))
+const https = vi.hoisted(() => ({ request: vi.fn() }))
+vi.mock('node:dns/promises', () => dns)
+vi.mock('node:https', () => https)
+import { postWebhook } from './webhook-transport.js'
+afterEach(() => vi.resetAllMocks())
+describe('pinned webhook POST transport', () => {
+  it('rejects mixed DNS answers and rebinding on subsequent attempts without opening a socket', async () => {
+    dns.lookup.mockResolvedValue([{ address: '93.184.216.34', family: 4 }, { address: '127.0.0.1', family: 4 }])
+    await expect(postWebhook('https://hooks.example.com', 'secret', '{}', 'event', new AbortController().signal)).rejects.toThrow('INVALID_DESTINATION')
+    expect(https.request).not.toHaveBeenCalled()
+    dns.lookup.mockResolvedValue([{ address: '::ffff:10.0.0.1', family: 6 }])
+    await expect(postWebhook('https://hooks.example.com', 'secret', '{}', 'event', new AbortController().signal)).rejects.toThrow('INVALID_DESTINATION')
+    expect(dns.lookup).toHaveBeenCalledTimes(2)
+  })
+  it('pins the socket lookup, keeps TLS hostname, signs POST bytes and never follows redirects', async () => {
+    dns.lookup.mockResolvedValue([{ address: '93.184.216.34', family: 4 }])
+    const sent: string[] = []
+    https.request.mockImplementation((url, options, callback) => {
+      expect(url.hostname).toBe('hooks.example.com')
+      expect(options.method).toBe('POST')
+      expect(options.agent).toBe(false)
+      expect(options.rejectUnauthorized).not.toBe(false)
+      const selected = vi.fn()
+      options.lookup('hooks.example.com', {}, selected)
+      expect(selected).toHaveBeenCalledWith(null, '93.184.216.34', 4)
+      expect(options.headers['x-evofw-signature']).toMatch(/^v1=[a-f0-9]{64}$/)
+      expect(options.headers['x-evofw-event-id']).toBe('event')
+      const request = new EventEmitter() as EventEmitter & { end: (body: string) => void }
+      request.end = body => {
+        sent.push(body)
+        const response = Object.assign(new EventEmitter(), { statusCode: 302 })
+        callback(response)
+        queueMicrotask(() => response.emit('end'))
+      }
+      return request
+    })
+    await expect(postWebhook('https://hooks.example.com/path', 'secret', '{"id":1}', 'event', new AbortController().signal)).rejects.toThrow('HTTP_302')
+    expect(sent).toEqual(['{"id":1}'])
+    expect(https.request).toHaveBeenCalledTimes(1)
+  })
+  it('stops a DNS lookup when shutdown aborts', async () => {
+    dns.lookup.mockImplementation(() => new Promise(() => {}))
+    const controller = new AbortController()
+    const pending = postWebhook('https://hooks.example.com', 'secret', '{}', 'event', controller.signal)
+    controller.abort()
+    await expect(pending).rejects.toThrow('DELIVERY_TIMEOUT')
+    expect(https.request).not.toHaveBeenCalled()
+  })
+})

apps/api/src/services/webhook-transport.ts

@@ -0,0 +1,43 @@
+import { lookup } from 'node:dns/promises'
+import { request } from 'node:https'
+import { isIP, type LookupFunction } from 'node:net'
+import { createHmac } from 'node:crypto'
+import { isPublicSourceAddress } from './safe-fetch.js'
+
+export function validateWebhookUrl(value: string): URL {
+  const url = new URL(value)
+  const host = url.hostname.replace(/^\[|\]$/g, '')
+  if (url.protocol !== 'https:' || url.username || url.password || url.hash || (url.port && url.port !== '443') || !host || (isIP(host) && !isPublicSourceAddress(host))) throw new Error('INVALID_DESTINATION')
+  return url
+}
+export function webhookSignature(secret: string, timestamp: string, body: string): string {
+  return `v1=${createHmac('sha256', secret).update(`${timestamp}.${body}`).digest('hex')}`
+}
+/** No redirects, proxy environment, shared sockets or configurable headers. */
+export async function postWebhook(destination: string, secret: string, body: string, eventId: string, parentSignal: AbortSignal): Promise<void> {
+  const url = validateWebhookUrl(destination)
+  const signal = AbortSignal.any([parentSignal, AbortSignal.timeout(8000)])
+  const host = url.hostname.replace(/^\[|\]$/g, '')
+  const addresses = await new Promise<{ address: string; family: number }[]>((resolve, reject) => {
+    const aborted = () => reject(new Error('DELIVERY_TIMEOUT'))
+    signal.addEventListener('abort', aborted, { once: true })
+    if (signal.aborted) aborted()
+    lookup(host, { all: true, verbatim: true }).then(resolve, reject).finally(() => signal.removeEventListener('abort', aborted))
+  })
+  if (!addresses.length || addresses.some(item => !isPublicSourceAddress(item.address))) throw new Error('INVALID_DESTINATION')
+  const chosen = addresses[0]!
+  const timestamp = String(Math.floor(Date.now() / 1000))
+  const pinned = ((_hostname: string, _options: unknown, done: (error: Error | null, address: string, family: number) => void) => done(null, chosen.address, chosen.family)) as LookupFunction
+  await new Promise<void>((resolve, reject) => {
+    const req = request(url, { method: 'POST', agent: false, family: chosen.family, lookup: pinned, signal,
+      headers: { 'content-type': 'application/json', 'content-length': Buffer.byteLength(body), 'x-evofw-event-id': eventId, 'x-evofw-timestamp': timestamp, 'x-evofw-signature': webhookSignature(secret, timestamp, body) },
+    }, response => {
+      let bytes = 0
+      response.on('data', (chunk: Buffer) => { bytes += chunk.length; if (bytes > 65536) response.destroy(new Error('RESPONSE_TOO_LARGE')) })
+      response.on('error', reject)
+      response.on('end', () => response.statusCode && response.statusCode >= 200 && response.statusCode < 300 ? resolve() : reject(new Error(`HTTP_${response.statusCode ?? 0}`)))
+    })
+    req.on('error', reject)
+    req.end(body)
+  })
+}

apps/api/src/services/webhooks-routes.test.ts

@@ -0,0 +1,30 @@
+import { describe, expect, it } from 'vitest'
+import { buildApp } from '../app.js'
+import { loadConfig } from '../config.js'
+const body = { name: 'test', url: 'https://hooks.example.com', secret: 's'.repeat(32), events: ['source.error'] }
+describe('webhook routes', () => {
+  it('requires settings admin and redacts secrets from all endpoint responses', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: true, secretKey: 'k'.repeat(32), staticDir: null, logLevel: 'silent' } })
+    try {
+      const token = (permissions: string[]) => app.jwt.sign({ sub: 'tester', apps: ['fw'], permissions, iss: loadConfig().authIssuer })
+      const reader = { authorization: `Bearer ${token(['fw:settings:read'])}` }
+      expect((await app.inject({ url: '/api/v1/webhooks', headers: reader })).statusCode).toBe(403)
+      const admin = { authorization: `Bearer ${token(['fw:settings:admin'])}` }
+      const created = await app.inject({ method: 'POST', url: '/api/v1/webhooks', headers: admin, payload: body })
+      expect(created.statusCode).toBe(201)
+      expect(created.body).not.toContain(body.secret)
+      expect(created.json().secret_configured).toBe(true)
+      expect((await app.inject({ url: '/api/v1/webhooks', headers: admin })).body).not.toContain(body.secret)
+      expect((await app.inject({ url: `/api/v1/webhooks/${created.json().id}/deliveries`, headers: admin })).json()).toEqual({ items: [], total: 0 })
+    } finally { await app.close() }
+  })
+  it('returns a clear 503 on create/update without storage key', async () => {
+    const app = await buildApp({ memory: true, config: { ...loadConfig(), authRequired: false, secretKey: null, staticDir: null, logLevel: 'silent' } })
+    try {
+      const result = await app.inject({ method: 'POST', url: '/api/v1/webhooks', payload: body })
+      expect(result.statusCode).toBe(503)
+      expect(result.json().error.code).toBe('WEBHOOK_KEY_REQUIRED')
+      expect((await app.inject({ method: 'PATCH', url: '/api/v1/webhooks/unknown', payload: { name: 'new' } })).statusCode).toBe(503)
+    } finally { await app.close() }
+  })
+})

apps/api/src/services/webhooks.test.ts

@@ -0,0 +1,114 @@
+import { describe, expect, it, vi } from 'vitest'
+import { createMemoryDb, runMigrations } from '@evofw/db'
+import { Webhooks } from './webhooks.js'
+import { loadConfig } from '../config.js'
+import { validateWebhookUrl, webhookSignature, postWebhook } from './webhook-transport.js'
+import type { WebhookBody } from '@evofw/shared'
+
+const body: WebhookBody = { name: 'Alerts', url: 'https://hooks.example.com/events', secret: 's'.repeat(32), events: ['agent.offline', 'source.error'], enabled: true }
+function fixture(key: string | null = 'k'.repeat(32), transport = vi.fn(async (..._args: Parameters<typeof postWebhook>) => {})) {
+  const { sqlite } = createMemoryDb(); runMigrations(sqlite)
+  const worker = new Webhooks(sqlite, { ...loadConfig(), secretKey: key }, transport)
+  return { sqlite, worker, transport }
+}
+describe('durable webhooks', () => {
+  it.each(['destination', 'disable'] as const)('does not send cached canceled deliveries after endpoint %s changes during an active request', async change => {
+    let entered!: () => void, finish!: () => void
+    const started = new Promise<void>(resolve => { entered = resolve })
+    const paused = new Promise<void>(resolve => { finish = resolve })
+    const transport = vi.fn(async (..._args: Parameters<typeof postWebhook>) => { entered(); await paused })
+    const { sqlite, worker } = fixture('k'.repeat(32), transport)
+    const now = Date.now()
+    try {
+      const endpoint = worker.create(body)
+      for (let i = 0; i < 3; i++) worker.observe('agent.offline', `queued-${i}`, true, new Date(now).toISOString(), now)
+      const active = worker.tick(now)
+      await started
+      worker.update(endpoint.id, change === 'disable' ? { enabled: false } : { url: 'https://new.example.com', secret: 'new'.repeat(16) })
+      finish()
+      await active
+      expect(transport).toHaveBeenCalledTimes(1)
+      expect(transport.mock.calls[0]?.[0]).toBe(body.url)
+      expect(worker.stats().pending).toBe(0)
+    } finally { finish(); await worker.close(); sqlite.close() }
+  })
+  it('requires a strong configured encryption key and never returns/stores plaintext secrets', async () => {
+    const { sqlite, worker } = fixture()
+    try {
+      const endpoint = worker.create(body)
+      expect(JSON.stringify(endpoint)).not.toContain(body.secret)
+      const cipher = sqlite.prepare('SELECT secret_cipher FROM webhook_endpoints').get() as { secret_cipher: string }
+      expect(cipher.secret_cipher).not.toContain(body.secret)
+      const missing = new Webhooks(sqlite, { ...loadConfig(), secretKey: null })
+      expect(() => missing.create(body)).toThrow('EVOFW_SECRET_KEY')
+      expect(() => missing.update(endpoint.id, { name: 'new' })).toThrow('EVOFW_SECRET_KEY')
+      expect(missing.list().configured).toBe(false)
+      const weak = new Webhooks(sqlite, { ...loadConfig(), secretKey: 'short' })
+      expect(() => weak.create(body)).toThrow('EVOFW_SECRET_KEY')
+    } finally { await worker.close(); sqlite.close() }
+  })
+  it('persists exact retry bytes, cooldown transitions and recovery across worker restart', async () => {
+    const failing = vi.fn(async (..._args: Parameters<typeof postWebhook>) => { throw new Error('https://secret:password@internal/private') })
+    const { sqlite, worker } = fixture('k'.repeat(32), failing)
+    const now = Date.now()
+    try {
+      const endpoint = worker.create(body)
+      // Use a source row so bounded scan preserves the transition across restart.
+      sqlite.prepare("INSERT INTO ip_lists(id,name,type,last_error,updated_at) VALUES ('source','Source','json_url','private failure',?)").run(new Date(now).toISOString())
+      await worker.tick(now)
+      expect(worker.stats().pending).toBe(1)
+      expect((worker.deliveries(endpoint.id).items[0] as { last_error: string }).last_error).toBe('DELIVERY_FAILED')
+      await worker.close()
+      const succeeds = vi.fn(async (..._args: Parameters<typeof postWebhook>) => {})
+      const restarted = new Webhooks(sqlite, { ...loadConfig(), secretKey: 'k'.repeat(32) }, succeeds)
+      await restarted.tick(now + 10_000)
+      expect(succeeds).toHaveBeenCalledTimes(1)
+      expect(succeeds.mock.calls[0]?.slice(2, 4)).toEqual(failing.mock.calls[0]?.slice(2, 4))
+      expect(restarted.stats().pending).toBe(0)
+      sqlite.prepare("UPDATE ip_lists SET last_error=NULL,refreshed_at=? WHERE id='source'").run(new Date(now + 20_000).toISOString())
+      await restarted.tick(now + 20_000)
+      expect(succeeds).toHaveBeenCalledTimes(1)
+      await restarted.tick(now + 301_000)
+      expect(succeeds).toHaveBeenCalledTimes(2)
+      expect(JSON.parse(String(succeeds.mock.calls[1]?.[2])).state).toBe('healthy')
+      await restarted.close()
+    } finally { await worker.close(); sqlite.close() }
+  })
+  it('bounds endpoints/outbox/deadletters and cancels queued deliveries on rotation/delete', async () => {
+    const { sqlite, worker } = fixture()
+    const now = Date.now()
+    try {
+      const endpoint = worker.create(body)
+      for (let i = 0; i < 1002; i++) worker.observe('agent.offline', `a${i}`, true, new Date(now).toISOString(), now)
+      expect(worker.stats()).toEqual({ pending: 1000, dead: 0, dropped: 2 })
+      worker.update(endpoint.id, { secret: 'n'.repeat(32) })
+      expect(worker.stats().pending).toBe(0)
+      for (let i = 0; i < 19; i++) worker.create({ ...body, name: `endpoint${i}` })
+      expect(() => worker.create(body)).toThrow('At most 20')
+      worker.observe('agent.offline', 'test', true, new Date(now).toISOString(), now)
+      worker.remove(endpoint.id)
+      expect(sqlite.prepare('SELECT count(*) AS n FROM webhook_deliveries WHERE endpoint_id=?').get(endpoint.id)).toEqual({ n: 0 })
+    } finally { await worker.close(); sqlite.close() }
+  })
+  it('dead-letters expired delivery and retains at most 100 terminal rows', async () => {
+    const transport = vi.fn(async (..._args: Parameters<typeof postWebhook>) => { throw new Error('failure') })
+    const { sqlite, worker } = fixture('k'.repeat(32), transport)
+    const now = Date.now()
+    try {
+      worker.create(body)
+      for (let i = 0; i < 110; i++) worker.observe('agent.offline', `a${i}`, true, new Date(now).toISOString(), now)
+      sqlite.prepare("UPDATE webhook_deliveries SET status='dead'").run()
+      await worker.tick(now)
+      expect(worker.stats().dead).toBe(100)
+      expect(transport).not.toHaveBeenCalled()
+    } finally { await worker.close(); sqlite.close() }
+  })
+})
+describe('webhook signing and destination policy', () => {
+  it('rejects local/private/mapped destinations, non-HTTPS, non443 and credentials', () => {
+    for (const url of ['http://example.com', 'https://127.0.0.1', 'https://[::1]', 'https://[::ffff:127.0.0.1]', 'https://10.0.0.1', 'https://user:[email protected]', 'https://example.com:8443', 'https://example.com/#token']) expect(() => validateWebhookUrl(url)).toThrow()
+    expect(validateWebhookUrl(body.url).hostname).toBe('hooks.example.com')
+    expect(webhookSignature('secret', '100', '{}')).toMatch(/^v1=[0-9a-f]{64}$/)
+    expect(webhookSignature('secret', '100', '{}')).not.toBe(webhookSignature('secret', '101', '{}'))
+  })
+})

apps/api/src/services/webhooks.ts

@@ -0,0 +1,161 @@
+import type { Sqlite } from '@evofw/db'
+import { createCipheriv, createDecipheriv, createHash, randomBytes, randomUUID } from 'node:crypto'
+import type { AppConfig } from '../config.js'
+import { AppError } from '../plugins/error-handler.js'
+import { postWebhook, validateWebhookUrl } from './webhook-transport.js'
+import type { WebhookBody, WebhookEndpoint, WebhookEventType } from '@evofw/shared'
+
+type EndpointRow = { id: string; name: string; url: string; enabled: number; generation: number; events_json: string; secret_cipher: string; created_at: string; updated_at: string }
+type DeliveryRow = { id: string; endpoint_id: string; payload_json: string; attempts: number; created_at: string }
+type Transport = typeof postWebhook
+const COOLDOWN = 300_000
+export const WEBHOOK_LIMITS = { endpoints: 20, pending: 1000 }
+
+/** Separate from legacy optional secret encryption: missing/weak keys fail closed. */
+export class Webhooks {
+  private running: Promise<void> | null = null
+  private abort = new AbortController()
+  private agentCursor = ''
+  private sourceCursor = ''
+  private droppedWarningAt = 0
+  constructor(private db: Sqlite, private config: AppConfig, private transport: Transport = postWebhook, private warning: () => void = () => {}) {}
+  configured(): boolean { return !!this.config.secretKey && this.config.secretKey.length >= 32 }
+  private key(): Buffer {
+    if (!this.configured()) throw new AppError('WEBHOOK_KEY_REQUIRED', 'Set EVOFW_SECRET_KEY to at least 32 characters before managing webhook endpoints', 503)
+    return createHash('sha256').update(this.config.secretKey!).digest()
+  }
+  private encrypt(secret: string, id: string): string {
+    const nonce = randomBytes(12), cipher = createCipheriv('aes-256-gcm', this.key(), nonce)
+    cipher.setAAD(Buffer.from(id))
+    const data = Buffer.concat([cipher.update(secret, 'utf8'), cipher.final()])
+    return ['v1', nonce.toString('base64'), cipher.getAuthTag().toString('base64'), data.toString('base64')].join(':')
+  }
+  private decrypt(value: string, id: string): string {
+    const [version, nonce, tag, data] = value.split(':')
+    if (version !== 'v1' || !nonce || !tag || !data) throw new Error('KEY_UNAVAILABLE')
+    const cipher = createDecipheriv('aes-256-gcm', this.key(), Buffer.from(nonce, 'base64'))
+    cipher.setAAD(Buffer.from(id)); cipher.setAuthTag(Buffer.from(tag, 'base64'))
+    return Buffer.concat([cipher.update(Buffer.from(data, 'base64')), cipher.final()]).toString('utf8')
+  }
+  private row(id: string): EndpointRow {
+    const row = this.db.prepare('SELECT * FROM webhook_endpoints WHERE id=?').get(id) as EndpointRow | undefined
+    if (!row) throw new AppError('NOT_FOUND', 'Webhook not found', 404)
+    return row
+  }
+  private dto(row: EndpointRow): WebhookEndpoint {
+    return { id: row.id, name: row.name, url: row.url, enabled: !!row.enabled, events: JSON.parse(row.events_json), secret_configured: true, created_at: row.created_at, updated_at: row.updated_at }
+  }
+  list() { return { items: (this.db.prepare('SELECT * FROM webhook_endpoints ORDER BY created_at,id LIMIT 20').all() as EndpointRow[]).map(row => this.dto(row)), configured: this.configured(), limits: WEBHOOK_LIMITS, queue: this.stats() } }
+  stats() {
+    const count = (state: string) => (this.db.prepare('SELECT count(*) AS n FROM webhook_deliveries WHERE status=?').get(state) as { n: number }).n
+    return { pending: count('pending'), dead: count('dead'), dropped: (this.db.prepare('SELECT dropped FROM webhook_metrics WHERE id=1').get() as { dropped: number }).dropped }
+  }
+  create(body: WebhookBody) {
+    this.key()
+    try { validateWebhookUrl(body.url) } catch { throw new AppError('VALIDATION_ERROR', 'Webhook requires a public HTTPS destination on port 443 without credentials or fragments', 400) }
+    const id = randomUUID(), now = new Date().toISOString()
+    this.db.transaction(() => {
+      if ((this.db.prepare('SELECT count(*) AS n FROM webhook_endpoints').get() as { n: number }).n >= WEBHOOK_LIMITS.endpoints) throw new AppError('WEBHOOK_LIMIT', 'At most 20 webhook endpoints', 409)
+      this.db.prepare('INSERT INTO webhook_endpoints(id,name,url,secret_cipher,enabled,events_json,created_at,updated_at) VALUES (?,?,?,?,?,?,?,?)').run(id, body.name, body.url, this.encrypt(body.secret, id), Number(body.enabled), JSON.stringify([...new Set(body.events)]), now, now)
+    })()
+    return this.dto(this.row(id))
+  }
+  update(id: string, body: Partial<WebhookBody>) {
+    this.key()
+    const row = this.row(id)
+    if (body.url !== undefined) try { validateWebhookUrl(body.url) } catch { throw new AppError('VALIDATION_ERROR', 'Invalid webhook destination', 400) }
+    this.db.transaction(() => {
+      // Changing delivery destination/key/subscriptions cancels pending bytes.
+      const deliveryChanged = body.secret !== undefined || body.url !== undefined || body.events !== undefined || body.enabled !== undefined
+      if (deliveryChanged) {
+        this.db.prepare('DELETE FROM webhook_deliveries WHERE endpoint_id=?').run(id)
+        this.db.prepare('UPDATE webhook_endpoints SET generation=generation+1 WHERE id=?').run(id)
+      }
+      if (body.events !== undefined || body.enabled === false) this.db.prepare('DELETE FROM webhook_transitions WHERE endpoint_id=?').run(id)
+      this.db.prepare('UPDATE webhook_endpoints SET name=?,url=?,secret_cipher=?,enabled=?,events_json=?,updated_at=? WHERE id=?').run(body.name ?? row.name, body.url ?? row.url, body.secret === undefined ? row.secret_cipher : this.encrypt(body.secret, id), body.enabled === undefined ? row.enabled : Number(body.enabled), body.events === undefined ? row.events_json : JSON.stringify([...new Set(body.events)]), new Date().toISOString(), id)
+    })()
+    return this.dto(this.row(id))
+  }
+  remove(id: string) { this.row(id); this.db.prepare('DELETE FROM webhook_endpoints WHERE id=?').run(id); return { ok: true } }
+  deliveries(id: string) {
+    this.row(id)
+    return { items: this.db.prepare('SELECT id,event_type,resource_id,status,attempts,next_attempt_at,created_at,last_error FROM webhook_deliveries WHERE endpoint_id=? ORDER BY created_at DESC,id LIMIT 100').all(id), total: (this.db.prepare('SELECT count(*) AS n FROM webhook_deliveries WHERE endpoint_id=?').get(id) as { n: number }).n }
+  }
+  /** Transition and event bytes are committed together, independent of ingestion. */
+  observe(eventType: WebhookEventType, resourceId: string, problem: boolean, observedAt: string, now = Date.now()) {
+    const endpoints = this.db.prepare('SELECT * FROM webhook_endpoints WHERE enabled=1 LIMIT 20').all() as EndpointRow[]
+    for (const endpoint of endpoints) {
+      if (!(JSON.parse(endpoint.events_json) as string[]).includes(eventType)) continue
+      this.db.transaction(() => {
+        const previous = this.db.prepare('SELECT state,last_emitted_at FROM webhook_transitions WHERE endpoint_id=? AND event_type=? AND resource_id=?').get(endpoint.id, eventType, resourceId) as { state: string; last_emitted_at: string | null } | undefined
+        const state = problem ? 'problem' : 'healthy'
+        if (previous?.state === state || (previous?.last_emitted_at && now - Date.parse(previous.last_emitted_at) < COOLDOWN)) return
+        const initialHealthy = !previous && !problem
+        const created = new Date(now).toISOString()
+        if (!initialHealthy) {
+          const id = randomUUID()
+          const payload = JSON.stringify({ version: 1, id, type: eventType, resource_id: resourceId, state, occurred_at: observedAt })
+          if (this.stats().pending >= WEBHOOK_LIMITS.pending || Buffer.byteLength(payload) > 16384) {
+            this.db.prepare('UPDATE webhook_metrics SET dropped=dropped+1 WHERE id=1').run()
+            if (now - this.droppedWarningAt >= COOLDOWN) { this.warning(); this.droppedWarningAt = now }
+          } else this.db.prepare('INSERT INTO webhook_deliveries(id,endpoint_id,event_type,resource_id,payload_json,endpoint_generation,next_attempt_at,created_at) VALUES (?,?,?,?,?,?,?,?)').run(id, endpoint.id, eventType, resourceId, payload, endpoint.generation, created, created)
+        }
+        this.db.prepare('INSERT INTO webhook_transitions(endpoint_id,event_type,resource_id,state,last_emitted_at) VALUES (?,?,?,?,?) ON CONFLICT(endpoint_id,event_type,resource_id) DO UPDATE SET state=excluded.state,last_emitted_at=excluded.last_emitted_at').run(endpoint.id, eventType, resourceId, state, initialHealthy ? null : created)
+      })()
+    }
+  }
+  private scan(now: number) {
+    type Agent = { id: string; last_seen_at: string | null; last_apply_at: string | null; last_apply_status: string | null; desired_hash: string | null; applied_hash: string | null; desired_hash_generation: number | null; policy_generation: number; applied_revision: number | null }
+    const agents = this.db.prepare("SELECT id,last_seen_at,last_apply_at,last_apply_status,desired_hash,applied_hash,desired_hash_generation,policy_generation,applied_revision FROM agents WHERE status='approved' AND id>? ORDER BY id LIMIT 100").all(this.agentCursor) as Agent[]
+    this.agentCursor = agents.length === 100 ? agents.at(-1)!.id : ''
+    for (const agent of agents) {
+      // A never-seen invited/approved device is unknown, not an offline incident.
+      if (agent.last_seen_at) this.observe('agent.offline', agent.id, now - Date.parse(agent.last_seen_at) > 180_000, agent.last_seen_at, now)
+      if (agent.last_apply_at) this.observe('agent.apply_failed', agent.id, agent.last_apply_status === 'error' || agent.last_apply_status === 'failed', agent.last_apply_at, now)
+      if (agent.applied_hash && agent.desired_hash && agent.desired_hash_generation === agent.policy_generation) this.observe('agent.drift', agent.id, agent.applied_revision === agent.policy_generation && agent.applied_hash !== agent.desired_hash, agent.last_apply_at ?? new Date(now).toISOString(), now)
+    }
+    const sources = this.db.prepare("SELECT id,refreshed_at,last_error,updated_at FROM ip_lists WHERE type<>'static' AND id>? ORDER BY id LIMIT 100").all(this.sourceCursor) as { id: string; refreshed_at: string | null; last_error: string | null; updated_at: string }[]
+    this.sourceCursor = sources.length === 100 ? sources.at(-1)!.id : ''
+    for (const source of sources) if (source.last_error || source.refreshed_at) this.observe('source.error', source.id, !!source.last_error, source.last_error ? source.updated_at : source.refreshed_at!, now)
+    this.db.prepare("DELETE FROM webhook_transitions WHERE (event_type LIKE 'agent.%' AND resource_id NOT IN (SELECT id FROM agents WHERE status='approved')) OR (event_type='source.error' AND resource_id NOT IN (SELECT id FROM ip_lists WHERE type<>'static'))").run()
+  }
+  tick(now = Date.now()): Promise<void> {
+    if (this.running) return this.running
+    if (this.abort.signal.aborted || !this.configured()) return Promise.resolve()
+    this.running = this.run(now).finally(() => { this.running = null })
+    return this.running
+  }
+  private async run(now: number) {
+    this.scan(now)
+    const rows = this.db.prepare("SELECT d.* FROM webhook_deliveries d JOIN webhook_endpoints e ON e.id=d.endpoint_id WHERE d.status='pending' AND d.next_attempt_at<=? AND e.enabled=1 ORDER BY d.next_attempt_at,d.id LIMIT 5").all(new Date(now).toISOString()) as DeliveryRow[]
+    for (const candidate of rows) {
+      if (this.abort.signal.aborted) break
+      // Earlier awaits allow an administrator to cancel/change an endpoint.
+      // Re-read eligibility and generation immediately before each request;
+      // cached batch payloads must never move to a new destination or secret.
+      const row = this.db.prepare("SELECT d.* FROM webhook_deliveries d JOIN webhook_endpoints e ON e.id=d.endpoint_id WHERE d.id=? AND d.status='pending' AND e.enabled=1 AND d.endpoint_generation=e.generation").get(candidate.id) as DeliveryRow | undefined
+      if (!row) continue
+      let error: string | null = null
+      if (now - Date.parse(row.created_at) >= 86400_000 || row.attempts >= 12) {
+        this.db.prepare("UPDATE webhook_deliveries SET status='dead',last_error='EXPIRED' WHERE id=?").run(row.id)
+        continue
+      }
+      try {
+        const endpoint = this.row(row.endpoint_id)
+        await this.transport(endpoint.url, this.decrypt(endpoint.secret_cipher, endpoint.id), row.payload_json, row.id, this.abort.signal)
+      } catch (caught) {
+        const message = caught instanceof Error ? caught.message : ''
+        error = /^(HTTP_[0-9]{3}|INVALID_DESTINATION|RESPONSE_TOO_LARGE|KEY_UNAVAILABLE)$/.test(message) ? message : 'DELIVERY_FAILED'
+      }
+      if (this.abort.signal.aborted) break
+      if (!error) this.db.prepare('DELETE FROM webhook_deliveries WHERE id=?').run(row.id)
+      else {
+        const attempts = row.attempts + 1
+        const wait = Math.min(3600_000, 5000 * 2 ** (attempts - 1)) * (0.8 + Math.random() * 0.4)
+        this.db.prepare('UPDATE webhook_deliveries SET attempts=?,status=?,next_attempt_at=?,last_error=? WHERE id=?').run(attempts, attempts >= 12 ? 'dead' : 'pending', new Date(now + wait).toISOString(), error, row.id)
+      }
+    }
+    this.db.prepare("DELETE FROM webhook_deliveries WHERE status='dead' AND id NOT IN (SELECT id FROM webhook_deliveries WHERE status='dead' ORDER BY created_at DESC,id LIMIT 100)").run()
+  }
+  async close() { this.abort.abort(); await this.running }
+}

apps/web/src/components/agents/add-agent-sheet.tsx

@@ -1,3 +1,4 @@
+import { invalidateAfterMutation } from '@/queries/invalidation'
 import { useEffect, useState } from 'react'
 import { useMutation, useQueryClient } from '@tanstack/react-query'
 import { useForm } from 'react-hook-form'
@@ -83,7 +84,7 @@ export function AddAgentSheet({ open, onOpenChange }: AddAgentSheetProps) {
     onSuccess: (link) => {
       setCreated(link)
       toast.success('Агент создан')
-      void qc.invalidateQueries({ queryKey: ['agents'] })
+      void invalidateAfterMutation(qc, 'agent')
     },
     onError: (e: Error) => toast.error(e.message),
   })
@@ -101,12 +102,13 @@ export function AddAgentSheet({ open, onOpenChange }: AddAgentSheetProps) {
           <SheetHeader className="shrink-0">
             <SheetTitle>Команда установки</SheetTitle>
             <SheetDescription>
-              Агент уже в списке (Invited). Скопируйте one-liner и выполните на
+              Агент уже в списке (Invited). Ссылка одноразовая, действует 24 часа и
-              хосте.
+              погашается при регистрации. Выполните команду на целевом хосте.
             </SheetDescription>
           </SheetHeader>
           <ScrollArea className="flex-1 px-4">
             <div className="grid auto-rows-min gap-4 py-2 pb-4">
+              <p className="text-muted-foreground text-sm">Истекает: {created.expires_at ? new Date(created.expires_at).toLocaleString() : 'неизвестно'}. Повторная установка по использованной ссылке недоступна.</p>
               <Field>
                 <FieldLabel>По id</FieldLabel>
                 <div className="flex flex-col gap-2">
@@ -167,7 +169,7 @@ export function AddAgentSheet({ open, onOpenChange }: AddAgentSheetProps) {
       open={open}
       onOpenChange={onOpenChange}
       title="Добавить агента"
-      description="Создайте агента и короткую install-ссылку."
+      description="Создайте агента и одноразовую install-ссылку на 24 часа."
       form={form}
       onSubmit={async (values) => {
         await create.mutateAsync(values)

apps/web/src/components/agents/agent-blocked-ips.tsx

@@ -1,3 +1,4 @@
+import { QueryState } from '@/components/query-state'
 import { useMemo } from 'react'
 import { useQuery } from '@tanstack/react-query'
 import {
@@ -137,6 +138,7 @@ export function AgentBlockedIps({ agentId, platform }: AgentBlockedIpsProps) {
         <FrameDescription>{description}</FrameDescription>
       </FrameHeader>
       <FramePanel className="p-0">
+        <QueryState isLoading={q.isPending} isError={q.isError} error={q.error} onRetry={() => void q.refetch()}>
         {q.isLoading ? (
           <div className="flex flex-col gap-2 p-4">
             <Skeleton className="h-8 w-full" />
@@ -172,6 +174,7 @@ export function AgentBlockedIps({ agentId, platform }: AgentBlockedIpsProps) {
             <DataGridTable />
           </DataGrid>
         )}
+        </QueryState>
       </FramePanel>
     </Frame>
   )

apps/web/src/components/agents/agent-blocked-ports.tsx

@@ -1,3 +1,4 @@
+import { QueryState } from '@/components/query-state'
 import { useMemo } from 'react'
 import { useQuery } from '@tanstack/react-query'
 import {
@@ -118,6 +119,7 @@ export function AgentBlockedPorts({ agentId }: AgentBlockedPortsProps) {
         </FrameDescription>
       </FrameHeader>
       <FramePanel className="p-0">
+        <QueryState isLoading={q.isPending} isError={q.isError} error={q.error} onRetry={() => void q.refetch()}>
         {q.isLoading ? (
           <div className="flex flex-col gap-2 p-4">
             <Skeleton className="h-8 w-full" />
@@ -149,6 +151,7 @@ export function AgentBlockedPorts({ agentId }: AgentBlockedPortsProps) {
             <DataGridTable />
           </DataGrid>
         )}
+        </QueryState>
       </FramePanel>
     </Frame>
   )