Hub stage 3: compare and sync app configs across servers
The hub keeps a baseline per app in /data/apps/<app>/baseline.json. Local fields (overrides.json) are left out of the sync and out of the matrix comparison. Each server gets the baseline plus its own values. Applying goes target picker, preview with per-server diffs and warnings, then apply. Apply refuses a server whose config changed since the preview, saves the old config to history, then reads the new one back to check it. Large config bodies are read only for signed-in callers. The App config dialog has Compare, Baseline and Local fields tabs and an empty state that offers an import. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
DenozordecandClaude Opus 5.5 committed
1 parent
6acd2d656d
commit
0e05039202
22 files changed
+2434
-58
No files matched your search
+32
-1
@@ -83,7 +83,7 @@ Public: `status`, `user/login`. After signing in: `user/session/get`, `user/logo
|
||||
`logs/query`, `logs/list`, `settings/get`, `admin/cluster/state` (synthesized), and the hub's own `hub/audit/list` (role with
|
||||
`audit.read`). They are listed in `routes.ts` with a kind (`read`, `write-one`, `write-all`) and the capability they need.
|
||||
|
||||
Everything else answers `403 The hub is read-only.` and is audited. `apps/config/get` and `apps/config/set` are not available yet.
|
||||
Everything else answers `403 The hub is read-only.` and is audited.
|
||||
|
||||
## App writes
|
||||
|
||||
@@ -99,6 +99,37 @@ Every write goes to a server with its `<ID>_ADMIN_TOKEN`; a server without one a
|
||||
the console's own single-server calls, forwarded to that server (store URLs checked as above). Refused without a node, and
|
||||
with `node=cluster`.
|
||||
- `apps/listStoreApps`: the first server's store, each app tagged with `installedOn`.
|
||||
- `apps/config/get` (read, `node` required) and `apps/config/set` (`apps.write`, `node` required; the old config is saved to the
|
||||
history first and the call is refused if that fails): the console's own per-server Config dialog.
|
||||
|
||||
## Config sync
|
||||
|
||||
Keeps one app's config the same on every server, except for the fields that must differ. Per app the hub stores, under
|
||||
`${HUB_DATA}/apps/<app>/`:
|
||||
|
||||
- `baseline.json`: `{ app, text, importedFrom?, updatedAt, updatedBy }`. The config as TEXT, so non-JSON configs work too.
|
||||
- `overrides.json`: `{ paths: [...], values: { <server>: { <path>: <value> } } }`. Paths are dotted (`db.path`,
|
||||
`networks.0.cidr`); at most 50; only for a JSON baseline.
|
||||
- `history/<time>__<server>__before-apply.txt` (and `before-uninstall`): the config a server had before the hub changed it
|
||||
(50 files per app kept).
|
||||
|
||||
A server's effective config is the baseline with that server's local values put in (JSON, pretty-printed with 2 spaces). A local
|
||||
path with no value for a server keeps the baseline value and is reported as a warning. Comparisons are by meaning (key order and
|
||||
spacing do not count) and leave the local fields out, so two servers that differ only there are "Same" in the matrix, and the
|
||||
matrix hash ignores them too.
|
||||
|
||||
| Endpoint | Method | Needs | Does |
|
||||
|---|---|---|---|
|
||||
| `hub/apps/config/compare?name=` | GET | read | Each server's live config, the baseline, the local fields, `inSync`, and the differences against the baseline and against the other servers. Config texts are returned here, to roles that can read. |
|
||||
| `hub/apps/overrides/current` | POST `{name, paths}` | `apps.write` | The servers' current values of those paths. |
|
||||
| `hub/apps/baseline/import` | POST `{name, server}` | `apps.write` | Baseline := that server's live config. |
|
||||
| `hub/apps/baseline/save` | POST `{name, text}` | `apps.write` | Saves the baseline (1 MB at most). Text that starts with `{` or `[` must be valid JSON; the parse error is returned. |
|
||||
| `hub/apps/overrides/save` | POST `{name, paths, values}` | `apps.write` | Saves the local fields. |
|
||||
| `hub/apps/config/preview` | POST `{name, targets}` | `apps.write` | Per target: current and effective text, the diff, warnings (a missing local value, a version older than the newest installed, no admin token) and `currentHash`. |
|
||||
| `hub/apps/config/apply` | POST `{name, targets, expect?}` | `apps.write` | Per target: re-reads the config; refuses if its hash differs from `expect[server]`; saves the old config to the history; `apps/config/set` with the admin token; reads it back and checks it. A server that rejects the config keeps its old one. |
|
||||
|
||||
Reading configs needs View on Apps for the `hub-reader` user (and `*_ADMIN_TOKEN` is used as a fallback if a server refuses the
|
||||
read token). The audit log records who did what, with the size and a short hash of a config, never the text.
|
||||
|
||||
Before an uninstall the app's config is saved to `${HUB_DATA}/apps/<app>/history/<time>__<server>__before-uninstall.txt`
|
||||
(50 files per app are kept). If that cannot be done the uninstall does not happen. Every target's result is written to the
|
||||
|
||||
@@ -0,0 +1,338 @@
|
||||
/*
|
||||
Config sync for one app across the servers: compare, baseline, local fields, preview, apply.
|
||||
|
||||
The baseline (kept in the hub) plus a server's local values is that server's effective config.
|
||||
Apply brings the selected servers to it: it re-reads the server, refuses if the config changed
|
||||
since the preview, saves the old config to the history, writes with the server's ADMIN token and
|
||||
reads the config back to check it. Config texts are returned to the caller (it is what they edit)
|
||||
but never written to the audit log.
|
||||
*/
|
||||
import type { HubConfig, HubServer } from './config.ts'
|
||||
import {
|
||||
MAX_CONFIG,
|
||||
cleanValues,
|
||||
diffConfigs,
|
||||
effectiveConfig,
|
||||
importLocalValues,
|
||||
looksLikeJson,
|
||||
parseJson,
|
||||
sameConfig,
|
||||
textHash,
|
||||
validatePaths,
|
||||
type Diff,
|
||||
} from './apps-sync.ts'
|
||||
import { Upstream, UpstreamError } from './upstream.ts'
|
||||
import { readBaseline, readOverrides, writeBaseline, writeHistory, writeOverrides, type Baseline, type Overrides } from './store.ts'
|
||||
|
||||
export class UserError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message)
|
||||
this.name = 'UserError'
|
||||
}
|
||||
}
|
||||
|
||||
const reason = (e: unknown): string => (e instanceof Error ? e.message : 'failed')
|
||||
|
||||
export interface Live {
|
||||
installed: boolean
|
||||
version?: string
|
||||
ok: boolean
|
||||
error?: string
|
||||
/** The server's config text; null when the app has none. */
|
||||
text: string | null
|
||||
}
|
||||
|
||||
export interface ServerView {
|
||||
id: string
|
||||
ok: boolean
|
||||
error?: string
|
||||
installed: boolean
|
||||
writable: boolean
|
||||
version?: string
|
||||
text: string | null
|
||||
hash: string
|
||||
effective?: string
|
||||
warnings: string[]
|
||||
/** Null without a baseline. */
|
||||
inSync: boolean | null
|
||||
/** What applying would change: this server's config to its effective config. */
|
||||
vsBaseline?: Diff
|
||||
/** This server's config relative to each other server's, local fields left out. */
|
||||
vsOthers: Record<string, Diff>
|
||||
}
|
||||
|
||||
export interface TargetPreview {
|
||||
server: string
|
||||
ok: boolean
|
||||
error?: string
|
||||
writable: boolean
|
||||
current: string | null
|
||||
currentHash: string
|
||||
effective: string
|
||||
changes: Diff
|
||||
unchanged: boolean
|
||||
warnings: string[]
|
||||
}
|
||||
|
||||
export interface TargetResult {
|
||||
server: string
|
||||
ok: boolean
|
||||
message?: string
|
||||
}
|
||||
|
||||
/* 9.10 is newer than 9.9: compared part by part, numerically. */
|
||||
export function compareVersions(a: string, b: string): number {
|
||||
const [x, y] = [a, b].map((v) => v.split('.').map((p) => Number.parseInt(p, 10) || 0))
|
||||
for (let i = 0; i < Math.max(x.length, y.length); i++) {
|
||||
const d = (x[i] ?? 0) - (y[i] ?? 0)
|
||||
if (d !== 0) return d
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
export class AppsConfig {
|
||||
#config: HubConfig
|
||||
#upstream: Upstream
|
||||
#now: () => number
|
||||
|
||||
constructor(config: HubConfig, upstream: Upstream, now: () => number = Date.now) {
|
||||
this.#config = config
|
||||
this.#upstream = upstream
|
||||
this.#now = now
|
||||
}
|
||||
|
||||
#server(id: string): HubServer {
|
||||
const found = this.#config.servers.find((s) => s.id === id)
|
||||
if (found === undefined) throw new UserError(`Unknown server: ${id}.`)
|
||||
return found
|
||||
}
|
||||
|
||||
#targets(ids: string[]): HubServer[] {
|
||||
const unique = [...new Set(ids)]
|
||||
if (unique.length === 0) throw new UserError('Choose at least one server.')
|
||||
return unique.map((id) => this.#server(id))
|
||||
}
|
||||
|
||||
/* The read token first, the admin token when the server refuses it. `fresh` skips the hub's cache. */
|
||||
async #readConfig(server: HubServer, name: string, fresh: boolean): Promise<string | null> {
|
||||
if (fresh) this.#upstream.invalidate(server.id)
|
||||
let response: unknown
|
||||
try {
|
||||
response = await this.#upstream.call(server, 'apps/config/get', { name })
|
||||
} catch (e) {
|
||||
if (!(e instanceof UpstreamError) || e.kind !== 'rejected' || server.adminToken === undefined) throw e
|
||||
response = await this.#upstream.call(server, 'apps/config/get', { name }, server.adminToken)
|
||||
}
|
||||
const config = (response as { config?: unknown } | null)?.config
|
||||
return typeof config === 'string' ? config : null
|
||||
}
|
||||
|
||||
async #live(server: HubServer, name: string, fresh: boolean): Promise<Live> {
|
||||
let version: string | undefined
|
||||
try {
|
||||
if (fresh) this.#upstream.invalidate(server.id)
|
||||
const list = (await this.#upstream.call(server, 'apps/list')) as { apps?: { name: string; version?: string }[] } | null
|
||||
const app = (list?.apps ?? []).find((a) => a.name === name)
|
||||
if (app === undefined) return { installed: false, ok: true, text: null }
|
||||
version = app.version
|
||||
} catch (e) {
|
||||
return { installed: false, ok: false, error: reason(e), text: null }
|
||||
}
|
||||
try {
|
||||
return { installed: true, ok: true, version, text: await this.#readConfig(server, name, false) }
|
||||
} catch (e) {
|
||||
return { installed: true, ok: false, version, error: reason(e), text: null }
|
||||
}
|
||||
}
|
||||
|
||||
async #liveAll(name: string, fresh: boolean): Promise<Map<string, Live>> {
|
||||
const entries = await Promise.all(this.#config.servers.map(async (s): Promise<[string, Live]> => [s.id, await this.#live(s, name, fresh)]))
|
||||
return new Map(entries)
|
||||
}
|
||||
|
||||
getBaseline(name: string): Baseline | null {
|
||||
return readBaseline(this.#config.dataDir, name)
|
||||
}
|
||||
|
||||
getOverrides(name: string): Overrides {
|
||||
return readOverrides(this.#config.dataDir, name)
|
||||
}
|
||||
|
||||
async compare(name: string): Promise<unknown> {
|
||||
const baseline = this.getBaseline(name)
|
||||
const overrides = this.getOverrides(name)
|
||||
const live = await this.#liveAll(name, false)
|
||||
const views: ServerView[] = this.#config.servers.map((s) => {
|
||||
const l = live.get(s.id) as Live
|
||||
const text = l.text
|
||||
const view: ServerView = {
|
||||
id: s.id,
|
||||
ok: l.ok,
|
||||
installed: l.installed,
|
||||
writable: s.adminToken !== undefined,
|
||||
text,
|
||||
hash: textHash(text ?? ''),
|
||||
warnings: [],
|
||||
inSync: null,
|
||||
vsOthers: {},
|
||||
}
|
||||
if (l.error !== undefined) view.error = l.error
|
||||
if (l.version !== undefined) view.version = l.version
|
||||
if (baseline !== null && l.installed && l.ok) {
|
||||
const effective = effectiveConfig(baseline.text, overrides, s.id)
|
||||
view.effective = effective.text
|
||||
view.warnings = effective.warnings
|
||||
view.inSync = sameConfig(text ?? '', effective.text)
|
||||
view.vsBaseline = diffConfigs(text ?? '', effective.text, [])
|
||||
}
|
||||
return view
|
||||
})
|
||||
for (const v of views) {
|
||||
if (!v.installed || !v.ok) continue
|
||||
for (const o of views) {
|
||||
if (o.id !== v.id && o.installed && o.ok) v.vsOthers[o.id] = diffConfigs(o.text ?? '', v.text ?? '', overrides.paths)
|
||||
}
|
||||
}
|
||||
return { name, baseline, overrides, servers: views }
|
||||
}
|
||||
|
||||
async importBaseline(name: string, serverId: string, user: string): Promise<{ detail: string }> {
|
||||
const server = this.#server(serverId)
|
||||
let text: string | null
|
||||
try {
|
||||
text = await this.#readConfig(server, name, true)
|
||||
} catch (e) {
|
||||
throw new UserError(`${serverId}: ${reason(e)}`)
|
||||
}
|
||||
if (text === null || text.trim() === '') throw new UserError(`${serverId} has no config for "${name}" to import.`)
|
||||
if (text.length > MAX_CONFIG) throw new UserError('The config is larger than 1 MB.')
|
||||
writeBaseline(this.#config.dataDir, { app: name, text, importedFrom: serverId, updatedAt: new Date(this.#now()).toISOString(), updatedBy: user })
|
||||
return { detail: `from ${serverId}, ${text.length} bytes, ${textHash(text)}` }
|
||||
}
|
||||
|
||||
saveBaseline(name: string, text: string, user: string): { detail: string } {
|
||||
if (text.length > MAX_CONFIG) throw new UserError('The baseline is larger than 1 MB.')
|
||||
if (looksLikeJson(text)) {
|
||||
const parsed = parseJson(text)
|
||||
if (!parsed.ok) throw new UserError(`The baseline is not valid JSON: ${parsed.error}`)
|
||||
}
|
||||
const old = this.getBaseline(name)
|
||||
writeBaseline(this.#config.dataDir, {
|
||||
app: name,
|
||||
text,
|
||||
...(old?.importedFrom === undefined ? {} : { importedFrom: old.importedFrom }),
|
||||
updatedAt: new Date(this.#now()).toISOString(),
|
||||
updatedBy: user,
|
||||
})
|
||||
return { detail: `${text.length} bytes, ${textHash(text)}` }
|
||||
}
|
||||
|
||||
saveOverrides(name: string, pathsRaw: unknown, valuesRaw: unknown): { detail: string } {
|
||||
let paths: string[]
|
||||
let values: Overrides['values']
|
||||
try {
|
||||
paths = validatePaths(pathsRaw)
|
||||
values = cleanValues(valuesRaw ?? {}, paths, this.#config.servers.map((s) => s.id))
|
||||
} catch (e) {
|
||||
throw new UserError(reason(e))
|
||||
}
|
||||
const baseline = this.getBaseline(name)
|
||||
if (paths.length > 0 && baseline !== null && (!looksLikeJson(baseline.text) || !parseJson(baseline.text).ok)) {
|
||||
throw new UserError('The baseline is plain text, so it cannot have local fields.')
|
||||
}
|
||||
if (JSON.stringify(values).length > MAX_CONFIG) throw new UserError('The local values are larger than 1 MB.')
|
||||
writeOverrides(this.#config.dataDir, name, { paths, values })
|
||||
return { detail: `${paths.length} local field(s)` }
|
||||
}
|
||||
|
||||
/* Reads each local path from the servers' live configs, for the "fill from current" button. */
|
||||
async currentLocalValues(name: string, pathsRaw: unknown): Promise<Overrides['values']> {
|
||||
let paths: string[]
|
||||
try {
|
||||
paths = validatePaths(pathsRaw)
|
||||
} catch (e) {
|
||||
throw new UserError(reason(e))
|
||||
}
|
||||
const live = await this.#liveAll(name, true)
|
||||
return importLocalValues(Object.fromEntries([...live].map(([id, l]) => [id, l.installed && l.ok ? l.text : null])), paths)
|
||||
}
|
||||
|
||||
async preview(name: string, ids: string[]): Promise<{ targets: TargetPreview[] }> {
|
||||
const targets = this.#targets(ids)
|
||||
const baseline = this.getBaseline(name)
|
||||
if (baseline === null) throw new UserError('This app has no baseline yet. Import one from a server first.')
|
||||
const overrides = this.getOverrides(name)
|
||||
const live = await this.#liveAll(name, true)
|
||||
const versions = [...live].filter(([, l]) => l.installed && l.version !== undefined).map(([id, l]) => ({ id, version: l.version as string }))
|
||||
const newest = versions.reduce<{ id: string; version: string } | null>((best, v) => (best === null || compareVersions(v.version, best.version) > 0 ? v : best), null)
|
||||
|
||||
return {
|
||||
targets: targets.map((s): TargetPreview => {
|
||||
const l = live.get(s.id) as Live
|
||||
const effective = effectiveConfig(baseline.text, overrides, s.id)
|
||||
const warnings = [...effective.warnings]
|
||||
if (s.adminToken === undefined) warnings.push('The hub has no admin token for this server, so it cannot write to it.')
|
||||
if (l.installed && l.version !== undefined && newest !== null && compareVersions(l.version, newest.version) < 0) {
|
||||
warnings.push(`Version ${l.version} here, ${newest.version} on ${newest.id}: versions differ, align versions first.`)
|
||||
}
|
||||
const current = l.text
|
||||
const base = {
|
||||
server: s.id,
|
||||
writable: s.adminToken !== undefined,
|
||||
current,
|
||||
currentHash: textHash(current ?? ''),
|
||||
effective: effective.text,
|
||||
warnings,
|
||||
}
|
||||
if (!l.ok) return { ...base, ok: false, error: l.error ?? 'did not answer', changes: { kind: 'text', lines: [], truncated: false, changed: 0 }, unchanged: false }
|
||||
if (!l.installed) return { ...base, ok: false, error: 'The app is not installed on this server.', changes: { kind: 'text', lines: [], truncated: false, changed: 0 }, unchanged: false }
|
||||
const changes = diffConfigs(current ?? '', effective.text, [])
|
||||
return { ...base, ok: true, changes, unchanged: sameConfig(current ?? '', effective.text) }
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
async apply(name: string, ids: string[], expect: Record<string, string>): Promise<{ results: TargetResult[]; details: Record<string, string> }> {
|
||||
const targets = this.#targets(ids)
|
||||
const baseline = this.getBaseline(name)
|
||||
if (baseline === null) throw new UserError('This app has no baseline yet. Import one from a server first.')
|
||||
const overrides = this.getOverrides(name)
|
||||
const details: Record<string, string> = {}
|
||||
|
||||
const results = await Promise.all(
|
||||
targets.map(async (s): Promise<TargetResult> => {
|
||||
if (s.adminToken === undefined) return { server: s.id, ok: false, message: 'no admin token configured' }
|
||||
try {
|
||||
const l = await this.#live(s, name, true)
|
||||
if (!l.ok) return { server: s.id, ok: false, message: l.error ?? 'did not answer' }
|
||||
if (!l.installed) return { server: s.id, ok: false, message: 'the app is not installed on this server' }
|
||||
const current = l.text ?? ''
|
||||
if (expect[s.id] !== undefined && expect[s.id] !== textHash(current)) {
|
||||
return { server: s.id, ok: false, message: 'the config changed on the server since the preview; run the preview again' }
|
||||
}
|
||||
const effective = effectiveConfig(baseline.text, overrides, s.id)
|
||||
if (sameConfig(current, effective.text)) return { server: s.id, ok: true, message: 'Already in sync; nothing was written.' }
|
||||
if (effective.text.length > MAX_CONFIG) return { server: s.id, ok: false, message: 'the config is larger than 1 MB' }
|
||||
|
||||
try {
|
||||
writeHistory(this.#config.dataDir, name, s.id, current, 'before-apply', this.#now())
|
||||
} catch (e) {
|
||||
return { server: s.id, ok: false, message: `the old config could not be saved first (${reason(e)}); nothing was changed` }
|
||||
}
|
||||
await this.#upstream.write(s, 'apps/config/set', {}, new URLSearchParams({ name, config: effective.text }))
|
||||
|
||||
const after = await this.#readConfig(s, name, true)
|
||||
details[s.id] = `${textHash(current)} to ${textHash(effective.text)}, ${effective.text.length} bytes`
|
||||
if (after === null || !sameConfig(after, effective.text)) {
|
||||
return { server: s.id, ok: false, message: 'the server accepted the config but reads back a different one' }
|
||||
}
|
||||
return { server: s.id, ok: true, message: 'Applied and verified.' }
|
||||
} catch (e) {
|
||||
return { server: s.id, ok: false, message: reason(e) }
|
||||
}
|
||||
}),
|
||||
)
|
||||
return { results, details }
|
||||
}
|
||||
}
|
||||
|
||||
+6
-2
@@ -6,6 +6,8 @@ hold secrets.
|
||||
*/
|
||||
import { createHash } from 'node:crypto'
|
||||
import type { HubConfig, HubServer } from './config.ts'
|
||||
import { comparable } from './apps-sync.ts'
|
||||
import { readOverrides } from './store.ts'
|
||||
import { jsonDiff } from './json-diff.ts'
|
||||
import { Upstream, UpstreamError } from './upstream.ts'
|
||||
|
||||
@@ -114,7 +116,7 @@ export async function buildMatrix(config: HubConfig, upstream: Upstream): Promis
|
||||
text = await readConfig(upstream, server, listed.name)
|
||||
if (text !== null && text !== '') {
|
||||
cell.configStatus = 'read'
|
||||
cell.configHash = hash(text)
|
||||
cell.configHash = hash(comparable(text, readOverrides(config.dataDir, listed.name).paths))
|
||||
}
|
||||
} catch (e) {
|
||||
cell.configStatus = 'unreadable'
|
||||
@@ -155,7 +157,9 @@ export async function buildMatrix(config: HubConfig, upstream: Upstream): Promis
|
||||
const versionSet = new Set(copies.map((c) => c.cell.version ?? ''))
|
||||
const versions = copies.length < 2 ? 'single' : versionSet.size === 1 ? 'same' : 'different'
|
||||
|
||||
const readable = copies.filter((c) => c.cell.configStatus !== 'unreadable').map((c) => c.text ?? '')
|
||||
// Fields marked local do not count: two servers that differ only there are the same.
|
||||
const localPaths = readOverrides(config.dataDir, name).paths
|
||||
const readable = copies.filter((c) => c.cell.configStatus !== 'unreadable').map((c) => comparable(c.text ?? '', localPaths))
|
||||
const hasUnknown = copies.some((c) => c.cell.configStatus === 'unreadable') || (warnings.length > 0 && copies.length < 2)
|
||||
const differs = new Set(readable).size > 1
|
||||
const configs: MatrixApp['configs'] = differs ? 'different' : hasUnknown ? 'unknown' : copies.length < 2 ? 'single' : 'same'
|
||||
|
||||
@@ -0,0 +1,529 @@
|
||||
// @vitest-environment node
|
||||
import http from 'node:http'
|
||||
import type { AddressInfo } from 'node:net'
|
||||
import { mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { parseConfig } from './config.ts'
|
||||
import { createHub } from './server.ts'
|
||||
import {
|
||||
comparable,
|
||||
diffConfigs,
|
||||
effectiveConfig,
|
||||
importLocalValues,
|
||||
lineDiff,
|
||||
parsePath,
|
||||
sameConfig,
|
||||
stripLocal,
|
||||
validatePaths,
|
||||
} from './apps-sync.ts'
|
||||
import { addUser, writeUsersFile } from './users.ts'
|
||||
|
||||
const json = (o: unknown): string => JSON.stringify(o, null, 2)
|
||||
|
||||
describe('effectiveConfig', () => {
|
||||
const baseline = json({ enable: true, db: { path: '/base.db', keep: 30 }, nets: [{ name: 'lan', cidr: '10.0.0.0/8' }, { name: 'wan' }] })
|
||||
|
||||
it('puts the server local values over the baseline, nested and in arrays', () => {
|
||||
const e = effectiveConfig(baseline, { paths: ['db.path', 'nets.0.cidr'], values: { a: { 'db.path': '/a.db', 'nets.0.cidr': '192.168.0.0/16' } } }, 'a')
|
||||
expect(JSON.parse(e.text)).toEqual({ enable: true, db: { path: '/a.db', keep: 30 }, nets: [{ name: 'lan', cidr: '192.168.0.0/16' }, { name: 'wan' }] })
|
||||
expect(e.warnings).toEqual([])
|
||||
expect(e.text).toBe(JSON.stringify(JSON.parse(e.text), null, 2))
|
||||
})
|
||||
|
||||
it('keeps the baseline value and warns when a server has no local value', () => {
|
||||
const e = effectiveConfig(baseline, { paths: ['db.path'], values: { a: {} } }, 'b')
|
||||
expect(JSON.parse(e.text).db.path).toBe('/base.db')
|
||||
expect(e.warnings[0]).toMatch(/No local value set for "db.path" on b/)
|
||||
})
|
||||
|
||||
it('leaves a field absent when it is local, not in the baseline and has no value', () => {
|
||||
const e = effectiveConfig(baseline, { paths: ['extra.deep'], values: {} }, 'a')
|
||||
expect(JSON.parse(e.text).extra).toBeUndefined()
|
||||
expect(e.warnings).toEqual([])
|
||||
})
|
||||
|
||||
it('creates the path when the local value is for a field the baseline lacks', () => {
|
||||
const e = effectiveConfig(baseline, { paths: ['extra.deep'], values: { a: { 'extra.deep': 5 } } }, 'a')
|
||||
expect(JSON.parse(e.text).extra).toEqual({ deep: 5 })
|
||||
})
|
||||
|
||||
it('uses a text baseline as it is', () => {
|
||||
const e = effectiveConfig('plain = 1\nother = 2', { paths: ['a'], values: {} }, 'a')
|
||||
expect(e.text).toBe('plain = 1\nother = 2')
|
||||
expect(e.warnings[0]).toMatch(/plain text/)
|
||||
expect(effectiveConfig('plain', { paths: [], values: {} }, 'a').warnings).toEqual([])
|
||||
})
|
||||
|
||||
it('never writes through __proto__', () => {
|
||||
const e = effectiveConfig('{}', { paths: ['__proto__.polluted'], values: { a: { '__proto__.polluted': true } } }, 'a')
|
||||
expect(({} as Record<string, unknown>).polluted).toBeUndefined()
|
||||
expect(e.warnings[0]).toMatch(/does not fit/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('paths', () => {
|
||||
it('parses dot paths and refuses odd ones', () => {
|
||||
expect(parsePath('a.b.0.c')).toEqual(['a', 'b', '0', 'c'])
|
||||
expect(parsePath('0.0.0.0/0')).toEqual(['0', '0', '0', '0/0'])
|
||||
for (const bad of ['', '.a', 'a..b', 'a b', 'a[0]', 'a,b']) expect(parsePath(bad)).toBeNull()
|
||||
expect(() => validatePaths(['a', 'a'])).toThrow(/twice/)
|
||||
expect(() => validatePaths(Array.from({ length: 51 }, (_, i) => `p${i}`))).toThrow(/50/)
|
||||
expect(() => validatePaths('a')).toThrow()
|
||||
})
|
||||
})
|
||||
|
||||
describe('comparisons that ignore local fields', () => {
|
||||
const a = json({ shared: 1, db: { path: '/a' }, list: [1, 2] })
|
||||
const b = json({ db: { path: '/b' }, list: [1, 2], shared: 1 })
|
||||
|
||||
it('treats two configs that differ only in local fields as the same', () => {
|
||||
expect(sameConfig(a, b)).toBe(false)
|
||||
expect(sameConfig(a, b, ['db.path'])).toBe(true)
|
||||
expect(comparable(a, ['db.path'])).toBe(comparable(b, ['db.path']))
|
||||
expect(comparable(a, [])).toBe(a)
|
||||
})
|
||||
|
||||
it('removes array items from the end first', () => {
|
||||
expect(stripLocal([1, 2, 3], ['0', '2'])).toEqual([2])
|
||||
})
|
||||
|
||||
it('diffs JSON by field and leaves local fields out', () => {
|
||||
const d = diffConfigs(json({ a: 1, b: 2, c: { d: 1 }, local: 1 }), json({ a: 1, b: 3, c: { d: 1, e: 2 }, f: 1, local: 2 }), ['local'])
|
||||
expect(d).toEqual({
|
||||
kind: 'json',
|
||||
changes: [
|
||||
{ path: 'b', type: 'changed', a: 2, b: 3 },
|
||||
{ path: 'c.e', type: 'added', a: undefined, b: 2 },
|
||||
{ path: 'f', type: 'added', a: undefined, b: 1 },
|
||||
],
|
||||
})
|
||||
expect(diffConfigs(json({ x: 1 }), json({}), []).kind).toBe('json')
|
||||
expect((diffConfigs(json({ x: 1 }), json({}), []) as { changes: { type: string }[] }).changes[0].type).toBe('removed')
|
||||
})
|
||||
|
||||
it('diffs text by line', () => {
|
||||
const d = diffConfigs('one\ntwo\nthree', 'one\n2\nthree\nfour')
|
||||
expect(d.kind).toBe('text')
|
||||
expect(d).toMatchObject({ changed: 3, truncated: false })
|
||||
expect((d as { lines: { op: string; text: string }[] }).lines).toEqual([
|
||||
{ op: 'same', text: 'one' },
|
||||
{ op: 'del', text: 'two' },
|
||||
{ op: 'add', text: '2' },
|
||||
{ op: 'same', text: 'three' },
|
||||
{ op: 'add', text: 'four' },
|
||||
])
|
||||
})
|
||||
|
||||
it('does not build a table for a huge text', () => {
|
||||
const big = Array.from({ length: 2500 }, (_, i) => `line ${i}`).join('\n')
|
||||
expect(lineDiff(big, big + '\nx')).toMatchObject({ truncated: true, lines: [] })
|
||||
})
|
||||
})
|
||||
|
||||
describe('importLocalValues', () => {
|
||||
it('reads each server own value of each path', () => {
|
||||
const values = importLocalValues({ a: json({ db: { path: '/a' } }), b: json({ db: { path: '/b' } }), c: 'not json', d: null }, ['db.path', 'missing'])
|
||||
expect(values).toEqual({ a: { 'db.path': '/a' }, b: { 'db.path': '/b' }, c: {}, d: {} })
|
||||
})
|
||||
})
|
||||
|
||||
/* ---- With fake servers ---- */
|
||||
|
||||
interface Fake {
|
||||
server: http.Server
|
||||
url: string
|
||||
apps: Record<string, { version: string; config: string | null }>
|
||||
calls: { path: string; token: string; method: string; form: URLSearchParams }[]
|
||||
rejectSet: string | null
|
||||
mangleSet: boolean
|
||||
denyRead: boolean
|
||||
}
|
||||
|
||||
function fake(): Promise<Fake> {
|
||||
return new Promise((resolve) => {
|
||||
const f: Fake = { server: http.createServer(), url: '', apps: {}, calls: [], rejectSet: null, mangleSet: false, denyRead: false }
|
||||
f.server.on('request', (req, res) => {
|
||||
const chunks: Buffer[] = []
|
||||
req.on('data', (c: Buffer) => chunks.push(c))
|
||||
req.on('end', () => {
|
||||
const url = new URL(req.url ?? '/', 'http://fake')
|
||||
const apiPath = url.pathname.replace(/^\/api\//, '')
|
||||
const token = url.searchParams.get('token') ?? ''
|
||||
const form = new URLSearchParams(Buffer.concat(chunks).toString('utf8'))
|
||||
f.calls.push({ path: apiPath, token, method: req.method ?? '', form })
|
||||
const name = url.searchParams.get('name') ?? form.get('name') ?? ''
|
||||
res.setHeader('content-type', 'application/json')
|
||||
const ok = (response: unknown = {}) => res.end(JSON.stringify({ status: 'ok', response }))
|
||||
const error = (m: string) => res.end(JSON.stringify({ status: 'error', errorMessage: m }))
|
||||
if (apiPath === 'apps/list') return ok({ apps: Object.entries(f.apps).map(([n, a]) => ({ name: n, version: a.version })) })
|
||||
if (apiPath === 'apps/config/get') {
|
||||
if (f.denyRead && !token.startsWith('admin-')) return error('Access was denied.')
|
||||
return f.apps[name] ? ok({ config: f.apps[name].config }) : error('not found')
|
||||
}
|
||||
if (apiPath === 'apps/config/set') {
|
||||
if (!token.startsWith('admin-')) return error('Access was denied.')
|
||||
if (f.rejectSet !== null) return error(f.rejectSet)
|
||||
f.apps[name].config = f.mangleSet ? '{"mangled":true}' : form.get('config')
|
||||
return ok()
|
||||
}
|
||||
ok()
|
||||
})
|
||||
})
|
||||
f.server.listen(0, '127.0.0.1', () => {
|
||||
f.url = `http://127.0.0.1:${(f.server.address() as AddressInfo).port}`
|
||||
resolve(f)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
let a: Fake
|
||||
let b: Fake
|
||||
let dataDir: string
|
||||
let staticDir: string
|
||||
let hub: http.Server | undefined
|
||||
let base: string
|
||||
const tokens: Record<string, string> = {}
|
||||
|
||||
const CFG_A = json({ enable: true, ttl: 30, dbPath: '/data/a.db' })
|
||||
const CFG_B = json({ enable: true, ttl: 60, dbPath: '/data/b.db' })
|
||||
|
||||
function reset(): void {
|
||||
a.calls = []
|
||||
b.calls = []
|
||||
for (const f of [a, b]) {
|
||||
f.rejectSet = null
|
||||
f.mangleSet = false
|
||||
f.denyRead = false
|
||||
}
|
||||
a.apps = { Blocker: { version: '1.0', config: CFG_A } }
|
||||
b.apps = { Blocker: { version: '1.0', config: CFG_B } }
|
||||
}
|
||||
|
||||
async function start(extra: Record<string, string> = {}): Promise<void> {
|
||||
hub?.close()
|
||||
hub = createHub(
|
||||
parseConfig({
|
||||
HUB_SERVERS: `a=${a.url},b=${b.url}`,
|
||||
A_TOKEN: 'read-a',
|
||||
B_TOKEN: 'read-b',
|
||||
A_ADMIN_TOKEN: 'admin-a',
|
||||
B_ADMIN_TOKEN: 'admin-b',
|
||||
HUB_DATA: dataDir,
|
||||
HUB_STATIC: staticDir,
|
||||
HUB_CACHE_SECONDS: '0',
|
||||
HUB_TIMEOUT_MS: '2000',
|
||||
...extra,
|
||||
}),
|
||||
)
|
||||
await new Promise<void>((resolve) => hub!.listen(0, '127.0.0.1', resolve))
|
||||
base = `http://127.0.0.1:${(hub.address() as AddressInfo).port}`
|
||||
for (const role of ['viewer', 'apps-admin', 'admin']) {
|
||||
const res = await fetch(`${base}/api/user/login`, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/x-www-form-urlencoded' },
|
||||
body: `user=${role}&pass=password-${role}`,
|
||||
})
|
||||
tokens[role] = ((await res.json()) as { token: string }).token
|
||||
}
|
||||
}
|
||||
|
||||
async function api(role: string, apiPath: string, init: { params?: Record<string, string>; json?: unknown; method?: string } = {}) {
|
||||
const headers: Record<string, string> = { authorization: `Bearer ${tokens[role]}` }
|
||||
let body: string | undefined
|
||||
if (init.json !== undefined) {
|
||||
headers['content-type'] = 'application/json'
|
||||
body = JSON.stringify(init.json)
|
||||
}
|
||||
const res = await fetch(`${base}/api/${apiPath}?${new URLSearchParams(init.params ?? {})}`, { method: init.method ?? (body === undefined ? 'GET' : 'POST'), headers, body })
|
||||
return { code: res.status, body: (await res.json()) as Record<string, any> }
|
||||
}
|
||||
|
||||
const dirOf = () => path.join(dataDir, 'apps', 'Blocker')
|
||||
const history = () => readdirSync(path.join(dirOf(), 'history'))
|
||||
const writes = (f: Fake) => f.calls.filter((c) => c.path === 'apps/config/set')
|
||||
|
||||
beforeAll(async () => {
|
||||
a = await fake()
|
||||
b = await fake()
|
||||
dataDir = mkdtempSync(path.join(tmpdir(), 'hub-sync-data-'))
|
||||
staticDir = mkdtempSync(path.join(tmpdir(), 'hub-sync-static-'))
|
||||
let users = addUser([], 'viewer', 'viewer', 'password-viewer')
|
||||
users = addUser(users, 'apps-admin', 'apps-admin', 'password-apps-admin')
|
||||
users = addUser(users, 'admin', 'admin', 'password-admin')
|
||||
writeUsersFile(dataDir, users)
|
||||
})
|
||||
|
||||
afterAll(() => {
|
||||
hub?.close()
|
||||
a.server.close()
|
||||
b.server.close()
|
||||
rmSync(dataDir, { recursive: true, force: true })
|
||||
rmSync(staticDir, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
beforeEach(async () => {
|
||||
reset()
|
||||
rmSync(path.join(dataDir, 'apps'), { recursive: true, force: true })
|
||||
rmSync(path.join(dataDir, 'audit.log'), { force: true })
|
||||
await start()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
hub?.close()
|
||||
})
|
||||
|
||||
const importFrom = (server: string) => api('apps-admin', 'hub/apps/baseline/import', { json: { name: 'Blocker', server } })
|
||||
const saveOverrides = (paths: string[], values: Record<string, Record<string, unknown>>) => api('apps-admin', 'hub/apps/overrides/save', { json: { name: 'Blocker', paths, values } })
|
||||
|
||||
describe('baseline and local fields', () => {
|
||||
it('has no baseline at first, and compare shows each server live config', async () => {
|
||||
const r = await api('viewer', 'hub/apps/config/compare', { params: { name: 'Blocker' } })
|
||||
expect(r.body.response.baseline).toBeNull()
|
||||
const [sa, sb] = r.body.response.servers
|
||||
expect(sa).toMatchObject({ id: 'a', installed: true, writable: true, version: '1.0', inSync: null, text: CFG_A })
|
||||
expect(sb.text).toBe(CFG_B)
|
||||
expect(sa.vsOthers.b.changes.map((c: any) => c.path)).toEqual(['ttl', 'dbPath'])
|
||||
})
|
||||
|
||||
it('imports a baseline from a server and stores it as text', async () => {
|
||||
const r = await importFrom('a')
|
||||
expect(r.body.response.baseline).toMatchObject({ app: 'Blocker', text: CFG_A, importedFrom: 'a', updatedBy: 'apps-admin' })
|
||||
const file = JSON.parse(readFileSync(path.join(dirOf(), 'baseline.json'), 'utf8'))
|
||||
expect(file.text).toBe(CFG_A)
|
||||
const cmp = (await api('viewer', 'hub/apps/config/compare', { params: { name: 'Blocker' } })).body.response
|
||||
expect(cmp.servers[0].inSync).toBe(true)
|
||||
expect(cmp.servers[1].inSync).toBe(false)
|
||||
expect(cmp.servers[1].vsBaseline.changes.map((c: any) => c.path)).toEqual(['ttl', 'dbPath'])
|
||||
})
|
||||
|
||||
it('refuses an import from a server with no config', async () => {
|
||||
a.apps.Blocker.config = null
|
||||
const r = await importFrom('a')
|
||||
expect(r.code).toBe(400)
|
||||
expect(r.body.errorMessage).toMatch(/no config/)
|
||||
})
|
||||
|
||||
it('validates a JSON baseline and reports the parse error', async () => {
|
||||
const bad = await api('apps-admin', 'hub/apps/baseline/save', { json: { name: 'Blocker', text: '{ "a": 1,, }' } })
|
||||
expect(bad.code).toBe(400)
|
||||
expect(bad.body.errorMessage).toMatch(/not valid JSON: .*position/)
|
||||
const text = await api('apps-admin', 'hub/apps/baseline/save', { json: { name: 'Blocker', text: 'plain = text' } })
|
||||
expect(text.body.response.baseline.text).toBe('plain = text')
|
||||
})
|
||||
|
||||
it('limits the baseline to 1 MB', async () => {
|
||||
const r = await api('apps-admin', 'hub/apps/baseline/save', { json: { name: 'Blocker', text: 'x'.repeat(1024 * 1024 + 1) } })
|
||||
expect(r.code).toBe(400)
|
||||
expect(r.body.errorMessage).toMatch(/1 MB/)
|
||||
})
|
||||
|
||||
it('saves local fields, rejects bad paths and a text baseline', async () => {
|
||||
await importFrom('a')
|
||||
const ok = await saveOverrides(['dbPath'], { a: { dbPath: '/data/a.db' }, b: { dbPath: '/data/b.db' } })
|
||||
expect(ok.body.response.overrides).toEqual({ paths: ['dbPath'], values: { a: { dbPath: '/data/a.db' }, b: { dbPath: '/data/b.db' } } })
|
||||
expect((await saveOverrides(['a b'], {})).code).toBe(400)
|
||||
expect((await saveOverrides(['dbPath'], { zzz: {} })).code).toBe(400)
|
||||
await api('apps-admin', 'hub/apps/baseline/save', { json: { name: 'Blocker', text: 'plain' } })
|
||||
const refused = await saveOverrides(['dbPath'], {})
|
||||
expect(refused.body.errorMessage).toMatch(/plain text/)
|
||||
})
|
||||
|
||||
it('fills local values from the current server configs', async () => {
|
||||
const r = await api('apps-admin', 'hub/apps/overrides/current', { json: { name: 'Blocker', paths: ['dbPath', 'ttl'] } })
|
||||
expect(r.body.response.values).toEqual({ a: { dbPath: '/data/a.db', ttl: 30 }, b: { dbPath: '/data/b.db', ttl: 60 } })
|
||||
})
|
||||
|
||||
it('makes the matrix say Same when servers differ only in local fields', async () => {
|
||||
const before = (await api('viewer', 'hub/apps/matrix')).body.response.apps[0]
|
||||
expect(before.configs).toBe('different')
|
||||
expect(before.diffCount).toBe(2)
|
||||
await saveOverrides(['ttl', 'dbPath'], {})
|
||||
const after = (await api('viewer', 'hub/apps/matrix')).body.response.apps[0]
|
||||
expect(after.configs).toBe('same')
|
||||
expect(after.servers.a.configHash).toBe(after.servers.b.configHash)
|
||||
await saveOverrides(['dbPath'], {})
|
||||
const some = (await api('viewer', 'hub/apps/matrix')).body.response.apps[0]
|
||||
expect(some).toMatchObject({ configs: 'different', diffCount: 1 })
|
||||
})
|
||||
|
||||
it('refuses writers-only calls to a viewer', async () => {
|
||||
for (const [p, json] of [
|
||||
['hub/apps/baseline/import', { name: 'Blocker', server: 'a' }],
|
||||
['hub/apps/baseline/save', { name: 'Blocker', text: '{}' }],
|
||||
['hub/apps/overrides/save', { name: 'Blocker', paths: [], values: {} }],
|
||||
['hub/apps/config/preview', { name: 'Blocker', targets: ['a'] }],
|
||||
['hub/apps/config/apply', { name: 'Blocker', targets: ['a'] }],
|
||||
] as const) {
|
||||
const r = await api('viewer', p, { json })
|
||||
expect([p, r.code]).toEqual([p, 403])
|
||||
}
|
||||
expect(writes(a)).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('preview and apply', () => {
|
||||
async function setup() {
|
||||
await importFrom('a')
|
||||
await saveOverrides(['dbPath'], { a: { dbPath: '/data/a.db' }, b: { dbPath: '/data/b.db' } })
|
||||
}
|
||||
|
||||
it('previews what would change on each target, with the local values in place', async () => {
|
||||
await setup()
|
||||
const r = await api('apps-admin', 'hub/apps/config/preview', { json: { name: 'Blocker', targets: ['a', 'b'] } })
|
||||
const [pa, pb] = r.body.response.targets
|
||||
expect(pa).toMatchObject({ server: 'a', ok: true, unchanged: true, writable: true })
|
||||
expect(pb).toMatchObject({ server: 'b', ok: true, unchanged: false, current: CFG_B })
|
||||
expect(pb.changes.changes.map((c: any) => [c.path, c.a, c.b])).toEqual([['ttl', 60, 30]])
|
||||
expect(JSON.parse(pb.effective)).toEqual({ enable: true, ttl: 30, dbPath: '/data/b.db' })
|
||||
expect(pb.currentHash).toMatch(/^[0-9a-f]{16}$/)
|
||||
})
|
||||
|
||||
it('warns about a missing local value, a version difference and a server without a token', async () => {
|
||||
await importFrom('a')
|
||||
await saveOverrides(['dbPath'], { a: { dbPath: '/data/a.db' } })
|
||||
b.apps.Blocker.version = '0.9'
|
||||
await start({ B_ADMIN_TOKEN: '' })
|
||||
const r = await api('apps-admin', 'hub/apps/config/preview', { json: { name: 'Blocker', targets: ['b'] } })
|
||||
const warnings: string[] = r.body.response.targets[0].warnings
|
||||
expect(r.body.response.targets[0].writable).toBe(false)
|
||||
expect(warnings.some((w) => /No local value set for "dbPath" on b/.test(w))).toBe(true)
|
||||
expect(warnings.some((w) => /align versions first/.test(w))).toBe(true)
|
||||
expect(warnings.some((w) => /no admin token/.test(w))).toBe(true)
|
||||
})
|
||||
|
||||
it('needs a baseline', async () => {
|
||||
const r = await api('apps-admin', 'hub/apps/config/preview', { json: { name: 'Blocker', targets: ['a'] } })
|
||||
expect(r.code).toBe(400)
|
||||
expect(r.body.errorMessage).toMatch(/no baseline/)
|
||||
})
|
||||
|
||||
it('applies: saves the old config, writes with the admin token, verifies', async () => {
|
||||
await setup()
|
||||
const preview = (await api('apps-admin', 'hub/apps/config/preview', { json: { name: 'Blocker', targets: ['a', 'b'] } })).body.response.targets
|
||||
const expectHashes = Object.fromEntries(preview.map((t: any) => [t.server, t.currentHash]))
|
||||
const r = await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['a', 'b'], expect: expectHashes } })
|
||||
const results = r.body.response.results
|
||||
expect(results.find((x: any) => x.server === 'a')).toMatchObject({ ok: true, message: expect.stringMatching(/Already in sync/) })
|
||||
expect(results.find((x: any) => x.server === 'b')).toEqual({ server: 'b', ok: true, message: 'Applied and verified.' })
|
||||
expect(JSON.parse(b.apps.Blocker.config as string)).toEqual({ enable: true, ttl: 30, dbPath: '/data/b.db' })
|
||||
expect(writes(a)).toHaveLength(0)
|
||||
const [w] = writes(b)
|
||||
expect(w.token).toBe('admin-b')
|
||||
expect(w.method).toBe('POST')
|
||||
expect(w.form.get('name')).toBe('Blocker')
|
||||
const files = history()
|
||||
expect(files).toHaveLength(1)
|
||||
expect(files[0]).toMatch(/__b__before-apply\.txt$/)
|
||||
expect(readFileSync(path.join(dirOf(), 'history', files[0]), 'utf8')).toBe(CFG_B)
|
||||
expect((await api('viewer', 'hub/apps/matrix')).body.response.apps[0].configs).toBe('same')
|
||||
})
|
||||
|
||||
it('refuses a target whose config changed since the preview', async () => {
|
||||
await setup()
|
||||
const preview = (await api('apps-admin', 'hub/apps/config/preview', { json: { name: 'Blocker', targets: ['b'] } })).body.response.targets
|
||||
b.apps.Blocker.config = json({ enable: false })
|
||||
const r = await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['b'], expect: { b: preview[0].currentHash } } })
|
||||
expect(r.body.response.results[0]).toMatchObject({ server: 'b', ok: false })
|
||||
expect(r.body.response.results[0].message).toMatch(/changed on the server since the preview/)
|
||||
expect(writes(b)).toHaveLength(0)
|
||||
expect(b.apps.Blocker.config).toBe(json({ enable: false }))
|
||||
})
|
||||
|
||||
it('reports a server that refuses the config and leaves it as it was', async () => {
|
||||
await setup()
|
||||
b.rejectSet = 'The config is invalid.'
|
||||
const r = await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['b'] } })
|
||||
expect(r.body.response.results[0]).toEqual({ server: 'b', ok: false, message: 'The config is invalid.' })
|
||||
expect(b.apps.Blocker.config).toBe(CFG_B)
|
||||
})
|
||||
|
||||
it('handles one target failing and the other succeeding', async () => {
|
||||
await setup()
|
||||
a.apps.Blocker.config = json({ enable: false })
|
||||
b.rejectSet = 'Disk full.'
|
||||
const r = await api('admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['a', 'b'] } })
|
||||
expect(r.body.response.results.map((x: any) => [x.server, x.ok])).toEqual([['a', true], ['b', false]])
|
||||
expect(JSON.parse(a.apps.Blocker.config as string).ttl).toBe(30)
|
||||
})
|
||||
|
||||
it('says so when the server reads back something else', async () => {
|
||||
await setup()
|
||||
b.mangleSet = true
|
||||
const r = await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['b'] } })
|
||||
expect(r.body.response.results[0].ok).toBe(false)
|
||||
expect(r.body.response.results[0].message).toMatch(/reads back a different one/)
|
||||
})
|
||||
|
||||
it('does not write to a server without an admin token', async () => {
|
||||
await setup()
|
||||
await start({ B_ADMIN_TOKEN: '' })
|
||||
const r = await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['b'] } })
|
||||
expect(r.body.response.results[0]).toEqual({ server: 'b', ok: false, message: 'no admin token configured' })
|
||||
expect(writes(b)).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('keeps config texts out of the audit log', async () => {
|
||||
await setup()
|
||||
await api('apps-admin', 'hub/apps/baseline/save', { json: { name: 'Blocker', text: '{ "bad": ,, "secret-baseline-text": 1 }' } })
|
||||
await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['a', 'b'] } })
|
||||
const raw = readFileSync(path.join(dataDir, 'audit.log'), 'utf8')
|
||||
for (const text of ['/data/a.db', '/data/b.db', 'secret-baseline-text', 'admin-a', 'admin-b']) expect(raw).not.toContain(text)
|
||||
const entries = (await api('admin', 'hub/audit/list', { params: { limit: '50' } })).body.response.entries as Record<string, string>[]
|
||||
const applied = entries.filter((e) => e.action === 'apps.config-apply')
|
||||
expect(applied).toHaveLength(2)
|
||||
expect(applied.find((e) => e.server === 'b')).toMatchObject({ user: 'apps-admin', target: 'Blocker', outcome: 'ok' })
|
||||
expect(applied.find((e) => e.server === 'b')?.detail).toMatch(/[0-9a-f]{16} to [0-9a-f]{16}/)
|
||||
expect(entries.some((e) => e.action === 'apps.baseline-import' && e.outcome === 'ok')).toBe(true)
|
||||
expect(entries.some((e) => e.action === 'apps.baseline-save' && e.outcome === 'refused' && e.detail === 'invalid baseline')).toBe(true)
|
||||
})
|
||||
|
||||
it('keeps at most 50 history files', async () => {
|
||||
await setup()
|
||||
mkdirSync(path.join(dirOf(), 'history'), { recursive: true })
|
||||
for (let i = 0; i < 55; i++) writeFileSync(path.join(dirOf(), 'history', `2020-01-01T00-00-${String(i).padStart(2, '0')}.000Z__b__before-apply.txt`), 'old')
|
||||
await api('apps-admin', 'hub/apps/config/apply', { json: { name: 'Blocker', targets: ['b'] } })
|
||||
expect(history()).toHaveLength(50)
|
||||
})
|
||||
})
|
||||
|
||||
describe('single-server config', () => {
|
||||
it('reads a config from one node with the read token and needs a node', async () => {
|
||||
const r = await api('viewer', 'apps/config/get', { params: { node: 'a', name: 'Blocker' } })
|
||||
expect(r.body.response.config).toBe(CFG_A)
|
||||
expect(a.calls.at(-1)?.token).toBe('read-a')
|
||||
expect((await api('viewer', 'apps/config/get', { params: { name: 'Blocker' } })).code).toBe(400)
|
||||
expect((await api('viewer', 'apps/config/get', { params: { node: 'cluster', name: 'Blocker' } })).code).toBe(400)
|
||||
})
|
||||
|
||||
it('sets a config on one node with the admin token, saving the old one first', async () => {
|
||||
const r = await api('apps-admin', 'apps/config/set', {
|
||||
params: { node: 'b' },
|
||||
method: 'POST',
|
||||
json: { name: 'Blocker', config: '{"enable":false}' },
|
||||
})
|
||||
expect(r.body.status).toBe('ok')
|
||||
const [w] = writes(b)
|
||||
expect(w.token).toBe('admin-b')
|
||||
expect(b.apps.Blocker.config).toBe('{"enable":false}')
|
||||
expect(history()).toHaveLength(1)
|
||||
expect(readFileSync(path.join(dirOf(), 'history', history()[0]), 'utf8')).toBe(CFG_B)
|
||||
const raw = readFileSync(path.join(dataDir, 'audit.log'), 'utf8')
|
||||
expect(raw).toContain('apps.config-set')
|
||||
expect(raw).not.toContain('"enable":false')
|
||||
})
|
||||
|
||||
it('refuses the write without a node, to a viewer, and when the old config cannot be saved', async () => {
|
||||
expect((await api('apps-admin', 'apps/config/set', { json: { name: 'Blocker', config: '{}' } })).code).toBe(400)
|
||||
expect((await api('viewer', 'apps/config/set', { params: { node: 'a' }, json: { name: 'Blocker', config: '{}' } })).code).toBe(403)
|
||||
a.denyRead = true
|
||||
a.apps.Blocker.config = 'unchanged'
|
||||
// The admin token can still read, so make the read fail by removing the app.
|
||||
delete a.apps.Blocker
|
||||
const r = await api('apps-admin', 'apps/config/set', { params: { node: 'a' }, json: { name: 'Blocker', config: '{}' } })
|
||||
expect(r.body.status).toBe('error')
|
||||
expect(r.body.errorMessage).toMatch(/nothing was changed/)
|
||||
expect(writes(a)).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,273 @@
|
||||
/*
|
||||
Pure functions behind the config sync: the effective config of a server (the baseline plus
|
||||
that server's local values), comparisons that leave the local fields out, and the diffs the
|
||||
console shows. No I/O here.
|
||||
*/
|
||||
import { createHash } from 'node:crypto'
|
||||
import { jsonDiff } from './json-diff.ts'
|
||||
import type { Overrides } from './store.ts'
|
||||
|
||||
export const MAX_CONFIG = 1024 * 1024
|
||||
export const MAX_PATHS = 50
|
||||
const MAX_LINES = 2000
|
||||
|
||||
type Json = unknown
|
||||
|
||||
export type Parsed = { ok: true; value: Json } | { ok: false; error: string }
|
||||
|
||||
export function parseJson(text: string): Parsed {
|
||||
try {
|
||||
return { ok: true, value: JSON.parse(text) }
|
||||
} catch (e) {
|
||||
return { ok: false, error: e instanceof Error ? e.message : 'invalid JSON' }
|
||||
}
|
||||
}
|
||||
|
||||
/** A config that starts like JSON must be JSON; anything else is treated as plain text. */
|
||||
export const looksLikeJson = (text: string): boolean => /^\s*[{[]/.test(text)
|
||||
|
||||
/* ---- Paths: `a.b.0.c`; a numeric segment indexes an array. ---- */
|
||||
|
||||
const SEGMENT = /^[^.\s,[\]]+$/
|
||||
|
||||
export function parsePath(path: string): string[] | null {
|
||||
if (path.length === 0 || path.length > 200) return null
|
||||
const segments = path.split('.')
|
||||
return segments.every((s) => SEGMENT.test(s)) ? segments : null
|
||||
}
|
||||
|
||||
export function validatePaths(paths: unknown): string[] {
|
||||
if (!Array.isArray(paths)) throw new Error('paths must be a list.')
|
||||
if (paths.length > MAX_PATHS) throw new Error(`At most ${MAX_PATHS} local fields.`)
|
||||
const seen = new Set<string>()
|
||||
for (const p of paths) {
|
||||
if (typeof p !== 'string' || parsePath(p) === null) throw new Error(`"${String(p)}" is not a field path such as a.b.0.c.`)
|
||||
if (seen.has(p)) throw new Error(`"${p}" is listed twice.`)
|
||||
seen.add(p)
|
||||
}
|
||||
return paths as string[]
|
||||
}
|
||||
|
||||
const isContainer = (v: Json): v is Record<string, Json> | Json[] => v !== null && typeof v === 'object'
|
||||
const isIndex = (s: string): boolean => /^\d+$/.test(s)
|
||||
|
||||
export function getAt(root: Json, segments: string[]): { found: boolean; value?: Json } {
|
||||
let at: Json = root
|
||||
for (const s of segments) {
|
||||
if (!isContainer(at)) return { found: false }
|
||||
if (Array.isArray(at)) {
|
||||
if (!isIndex(s) || Number(s) >= at.length) return { found: false }
|
||||
at = at[Number(s)]
|
||||
} else {
|
||||
if (!Object.hasOwn(at, s)) return { found: false }
|
||||
at = at[s]
|
||||
}
|
||||
}
|
||||
return { found: true, value: at }
|
||||
}
|
||||
|
||||
/* Sets a value, making the containers on the way when they are missing. Returns false if the path cannot be set. */
|
||||
export function setAt(root: Json, segments: string[], value: Json): boolean {
|
||||
let at: Json = root
|
||||
for (let i = 0; i < segments.length; i++) {
|
||||
const s = segments[i]
|
||||
const last = i === segments.length - 1
|
||||
if (!isContainer(at)) return false
|
||||
if (Array.isArray(at)) {
|
||||
if (!isIndex(s)) return false
|
||||
const index = Number(s)
|
||||
if (index > at.length) return false
|
||||
if (last) at[index] = value
|
||||
else {
|
||||
if (at[index] === undefined) at[index] = isIndex(segments[i + 1]) ? [] : {}
|
||||
at = at[index]
|
||||
}
|
||||
} else {
|
||||
if (s === '__proto__') return false
|
||||
if (last) at[s] = value
|
||||
else {
|
||||
if (!Object.hasOwn(at, s) || !isContainer(at[s])) at[s] = isIndex(segments[i + 1]) ? [] : {}
|
||||
at = at[s]
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
export function removeAt(root: Json, segments: string[]): void {
|
||||
const parent = getAt(root, segments.slice(0, -1))
|
||||
const last = segments[segments.length - 1]
|
||||
if (!parent.found || !isContainer(parent.value)) return
|
||||
if (Array.isArray(parent.value)) {
|
||||
if (isIndex(last) && Number(last) < parent.value.length) parent.value.splice(Number(last), 1)
|
||||
} else delete parent.value[last]
|
||||
}
|
||||
|
||||
const clone = <T>(value: T): T => structuredClone(value)
|
||||
|
||||
/* A copy of the value without the local fields. Removing array items shifts the rest, so the deepest and highest go first. */
|
||||
export function stripLocal(value: Json, paths: string[]): Json {
|
||||
const copy = clone(value)
|
||||
const sorted = paths
|
||||
.map((p) => parsePath(p))
|
||||
.filter((s): s is string[] => s !== null)
|
||||
.sort((a, b) => b.length - a.length || b.join('.').localeCompare(a.join('.'), undefined, { numeric: true }))
|
||||
for (const segments of sorted) removeAt(copy, segments)
|
||||
return copy
|
||||
}
|
||||
|
||||
function sortKeys(value: Json): Json {
|
||||
if (Array.isArray(value)) return value.map(sortKeys)
|
||||
if (isContainer(value)) return Object.fromEntries(Object.keys(value).sort().map((k) => [k, sortKeys((value as Record<string, Json>)[k])]))
|
||||
return value
|
||||
}
|
||||
|
||||
/*
|
||||
A string two configs can be compared by: for JSON, the config without the local fields with its keys
|
||||
sorted; for anything else, the text. Without local fields it is the text itself, as before.
|
||||
*/
|
||||
export function comparable(text: string, paths: string[]): string {
|
||||
if (paths.length === 0) return text
|
||||
const parsed = parseJson(text)
|
||||
if (!parsed.ok) return text
|
||||
return JSON.stringify(sortKeys(stripLocal(parsed.value, paths)))
|
||||
}
|
||||
|
||||
export const textHash = (text: string): string => createHash('sha256').update(text).digest('hex').slice(0, 16)
|
||||
|
||||
/* ---- The effective config of one server ---- */
|
||||
|
||||
export interface Effective {
|
||||
text: string
|
||||
warnings: string[]
|
||||
}
|
||||
|
||||
export function effectiveConfig(baselineText: string, overrides: Overrides, server: string): Effective {
|
||||
const parsed = parseJson(baselineText)
|
||||
if (!parsed.ok || !looksLikeJson(baselineText)) {
|
||||
const warnings = overrides.paths.length > 0 ? ['The baseline is plain text, so local fields cannot be applied; the baseline is used as it is.'] : []
|
||||
return { text: baselineText, warnings }
|
||||
}
|
||||
const value = clone(parsed.value)
|
||||
const warnings: string[] = []
|
||||
const mine = overrides.values[server] ?? {}
|
||||
for (const path of overrides.paths) {
|
||||
const segments = parsePath(path)
|
||||
if (segments === null) continue
|
||||
if (Object.hasOwn(mine, path)) {
|
||||
if (!setAt(value, segments, clone(mine[path]))) warnings.push(`Local field "${path}" does not fit the baseline structure and was skipped.`)
|
||||
} else if (getAt(value, segments).found) {
|
||||
warnings.push(`No local value set for "${path}" on ${server}; the baseline value is used.`)
|
||||
}
|
||||
// No local value and not in the baseline: nothing to remove, the field stays absent.
|
||||
}
|
||||
return { text: JSON.stringify(value, null, 2), warnings }
|
||||
}
|
||||
|
||||
/* The same config? JSON is compared by meaning (key order and spacing do not count), anything else by text. */
|
||||
export function sameConfig(a: string, b: string, paths: string[] = []): boolean {
|
||||
const [pa, pb] = [parseJson(a), parseJson(b)]
|
||||
if (pa.ok && pb.ok && looksLikeJson(a) && looksLikeJson(b)) return jsonDiff(stripLocal(pa.value, paths), stripLocal(pb.value, paths)).length === 0
|
||||
return a.replace(/\r\n/g, '\n').trimEnd() === b.replace(/\r\n/g, '\n').trimEnd()
|
||||
}
|
||||
|
||||
/* ---- Diffs ---- */
|
||||
|
||||
export interface FieldChange {
|
||||
path: string
|
||||
type: 'added' | 'removed' | 'changed'
|
||||
a?: Json
|
||||
b?: Json
|
||||
}
|
||||
|
||||
export interface LineOp {
|
||||
op: 'same' | 'add' | 'del'
|
||||
text: string
|
||||
}
|
||||
|
||||
export type Diff =
|
||||
| { kind: 'json'; changes: FieldChange[] }
|
||||
| { kind: 'text'; lines: LineOp[]; truncated: boolean; changed: number }
|
||||
|
||||
/* A to B: `added` means B has it and A does not. Local fields are left out of both. */
|
||||
export function diffConfigs(a: string, b: string, localPaths: string[] = []): Diff {
|
||||
const [pa, pb] = [parseJson(a), parseJson(b)]
|
||||
if (pa.ok && pb.ok && looksLikeJson(a) && looksLikeJson(b)) {
|
||||
const changes = jsonDiff(stripLocal(pa.value, localPaths), stripLocal(pb.value, localPaths)).map((d): FieldChange => {
|
||||
const type = d.a === undefined ? 'added' : d.b === undefined ? 'removed' : 'changed'
|
||||
return { path: d.path === '' ? '(whole config)' : d.path, type, a: d.a, b: d.b }
|
||||
})
|
||||
return { kind: 'json', changes }
|
||||
}
|
||||
return { kind: 'text', ...lineDiff(a, b) }
|
||||
}
|
||||
|
||||
export function lineDiff(a: string, b: string): { lines: LineOp[]; truncated: boolean; changed: number } {
|
||||
const [x, y] = [a.replace(/\r\n/g, '\n').split('\n'), b.replace(/\r\n/g, '\n').split('\n')]
|
||||
if (x.length > MAX_LINES || y.length > MAX_LINES) {
|
||||
const set = new Set(x)
|
||||
const changed = y.filter((l) => !set.has(l)).length + x.filter((l) => !new Set(y).has(l)).length
|
||||
return { lines: [], truncated: true, changed }
|
||||
}
|
||||
// Longest common subsequence table, filled from the end.
|
||||
const w = y.length + 1
|
||||
const table = new Uint16Array((x.length + 1) * w)
|
||||
for (let i = x.length - 1; i >= 0; i--) {
|
||||
for (let j = y.length - 1; j >= 0; j--) {
|
||||
table[i * w + j] = x[i] === y[j] ? table[(i + 1) * w + j + 1] + 1 : Math.max(table[(i + 1) * w + j], table[i * w + j + 1])
|
||||
}
|
||||
}
|
||||
const lines: LineOp[] = []
|
||||
let i = 0
|
||||
let j = 0
|
||||
while (i < x.length || j < y.length) {
|
||||
if (i < x.length && j < y.length && x[i] === y[j]) {
|
||||
lines.push({ op: 'same', text: x[i] })
|
||||
i++
|
||||
j++
|
||||
} else if (j < y.length && (i === x.length || table[i * w + j + 1] > table[(i + 1) * w + j])) {
|
||||
lines.push({ op: 'add', text: y[j++] })
|
||||
} else lines.push({ op: 'del', text: x[i++] })
|
||||
}
|
||||
return { lines, truncated: false, changed: lines.filter((l) => l.op !== 'same').length }
|
||||
}
|
||||
|
||||
export const diffIsEmpty = (d: Diff): boolean => (d.kind === 'json' ? d.changes.length === 0 : d.changed === 0)
|
||||
|
||||
/* ---- Local values ---- */
|
||||
|
||||
/*
|
||||
Fills each server's current value of each local path from its live config. A server whose config is
|
||||
not JSON, or that has no such field, gets no entry for it.
|
||||
*/
|
||||
export function importLocalValues(configs: Record<string, string | null>, paths: string[]): Overrides['values'] {
|
||||
const out: Overrides['values'] = {}
|
||||
for (const [server, text] of Object.entries(configs)) {
|
||||
out[server] = {}
|
||||
if (text === null) continue
|
||||
const parsed = parseJson(text)
|
||||
if (!parsed.ok) continue
|
||||
for (const path of paths) {
|
||||
const segments = parsePath(path)
|
||||
if (segments === null) continue
|
||||
const found = getAt(parsed.value, segments)
|
||||
if (found.found) out[server][path] = clone(found.value)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/* Local values for paths that are listed, from servers that exist; anything else is dropped. */
|
||||
export function cleanValues(values: unknown, paths: string[], servers: string[]): Overrides['values'] {
|
||||
if (values === null || typeof values !== 'object' || Array.isArray(values)) throw new Error('values must be an object of servers.')
|
||||
const out: Overrides['values'] = {}
|
||||
for (const [server, mine] of Object.entries(values as Record<string, unknown>)) {
|
||||
if (!servers.includes(server)) throw new Error(`Unknown server: ${server}.`)
|
||||
if (mine === null || typeof mine !== 'object' || Array.isArray(mine)) throw new Error(`The values of ${server} must be an object of paths.`)
|
||||
out[server] = {}
|
||||
for (const [path, value] of Object.entries(mine as Record<string, unknown>)) {
|
||||
if (paths.includes(path)) out[server][path] = value
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -499,9 +499,9 @@ describe('single-server writes', () => {
|
||||
expect(Buffer.from(parseMultipart(w.body, w.contentType).files[0].data).equals(Buffer.from(bytes))).toBe(true)
|
||||
})
|
||||
|
||||
it('keeps config get and set refused for now', async () => {
|
||||
it('needs a node for a config read or write', async () => {
|
||||
for (const p of ['apps/config/get', 'apps/config/set']) {
|
||||
expect((await api('admin', p, { params: { node: 'a', name: 'Blocker' } })).code).toBe(403)
|
||||
expect((await api('admin', p, { params: { name: 'Blocker' } })).code).toBe(400)
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
+19
-26
@@ -9,16 +9,17 @@ hub that changes anything, so the rules are strict:
|
||||
the uninstall is refused if that cannot be done;
|
||||
* every target's result is audited (the caller does that with `Audit`).
|
||||
*/
|
||||
import { createHash } from 'node:crypto'
|
||||
import { mkdirSync, readdirSync, renameSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import type { HubConfig, HubServer } from './config.ts'
|
||||
import type { UploadedFile } from './multipart.ts'
|
||||
import { MAX_CONFIG } from './apps-sync.ts'
|
||||
import { safeName, writeHistory } from './store.ts'
|
||||
import { Upstream } from './upstream.ts'
|
||||
|
||||
export { safeName }
|
||||
|
||||
export type StoreAction = 'install-store' | 'update-store' | 'uninstall'
|
||||
export type FileAction = 'install-file' | 'update-file'
|
||||
export type AppAction = StoreAction | FileAction
|
||||
export type AppAction = StoreAction | FileAction | 'config-set'
|
||||
|
||||
export interface TargetResult {
|
||||
server: string
|
||||
@@ -26,14 +27,13 @@ export interface TargetResult {
|
||||
message?: string
|
||||
}
|
||||
|
||||
export const KEEP_HISTORY = 50
|
||||
|
||||
const API: Record<AppAction, string> = {
|
||||
'install-store': 'apps/downloadAndInstall',
|
||||
'update-store': 'apps/downloadAndUpdate',
|
||||
uninstall: 'apps/uninstall',
|
||||
'install-file': 'apps/install',
|
||||
'update-file': 'apps/update',
|
||||
'config-set': 'apps/config/set',
|
||||
}
|
||||
|
||||
const DONE: Record<AppAction, string> = {
|
||||
@@ -42,6 +42,7 @@ const DONE: Record<AppAction, string> = {
|
||||
uninstall: 'Uninstalled.',
|
||||
'install-file': 'Installed from the file.',
|
||||
'update-file': 'Updated from the file.',
|
||||
'config-set': 'Config saved.',
|
||||
}
|
||||
|
||||
/** The audit/api action for an upstream write path, or null when it is not an app write. */
|
||||
@@ -50,13 +51,6 @@ export function actionForPath(apiPath: string): AppAction | null {
|
||||
return null
|
||||
}
|
||||
|
||||
/* A folder name for an app: letters, digits and . _ - only, with a short hash when anything had to change. */
|
||||
export function safeName(name: string): string {
|
||||
const cleaned = name.replace(/[^A-Za-z0-9._-]/g, '_').replace(/\.{2,}/g, '_').replace(/^\.+/, '_').slice(0, 80)
|
||||
if (cleaned === name && cleaned !== '') return cleaned
|
||||
return `${cleaned === '' ? '_' : cleaned}-${createHash('sha256').update(name).digest('hex').slice(0, 6)}`
|
||||
}
|
||||
|
||||
const reason = (e: unknown): string => (e instanceof Error ? e.message : 'failed')
|
||||
|
||||
export class AppsWriter {
|
||||
@@ -83,29 +77,19 @@ export class AppsWriter {
|
||||
}
|
||||
|
||||
/* Saves the app's current config next to the hub's data; throws if it cannot. */
|
||||
async saveHistory(server: HubServer, name: string): Promise<string> {
|
||||
async saveHistory(server: HubServer, name: string, kind: 'before-uninstall' | 'before-apply' = 'before-uninstall'): Promise<string> {
|
||||
if (server.adminToken === undefined) throw new Error('no admin token configured')
|
||||
this.#upstream.invalidate(server.id)
|
||||
const response = (await this.#upstream.call(server, 'apps/config/get', { name }, server.adminToken)) as { config?: unknown } | null
|
||||
const text = typeof response?.config === 'string' ? response.config : ''
|
||||
const dir = path.join(this.#config.dataDir, 'apps', safeName(name), 'history')
|
||||
mkdirSync(dir, { recursive: true })
|
||||
const stamp = new Date(this.#now()).toISOString().replace(/:/g, '-')
|
||||
const file = path.join(dir, `${stamp}__${safeName(server.id)}__before-uninstall.txt`)
|
||||
const temp = `${file}.${process.pid}.tmp`
|
||||
writeFileSync(temp, text, { mode: 0o600 })
|
||||
renameSync(temp, file)
|
||||
// Oldest first by name (the name starts with the time); keep the newest KEEP_HISTORY.
|
||||
const files = readdirSync(dir).filter((f) => f.endsWith('.txt')).sort()
|
||||
for (const old of files.slice(0, Math.max(0, files.length - KEEP_HISTORY))) rmSync(path.join(dir, old), { force: true })
|
||||
return file
|
||||
return writeHistory(this.#config.dataDir, name, server.id, text, kind, this.#now())
|
||||
}
|
||||
|
||||
/*
|
||||
One write on one server. `url` must already be checked against the store; `file` is the zip
|
||||
for the two file actions. Returns the server's response, or throws.
|
||||
*/
|
||||
async perform(server: HubServer, action: AppAction, name: string, url?: string, file?: UploadedFile): Promise<unknown> {
|
||||
async perform(server: HubServer, action: AppAction, name: string, url?: string, file?: UploadedFile, config?: string): Promise<unknown> {
|
||||
if (server.adminToken === undefined) throw new Error('no admin token configured')
|
||||
switch (action) {
|
||||
case 'install-store':
|
||||
@@ -118,6 +102,15 @@ export class AppsWriter {
|
||||
throw new Error(`the config could not be saved before the uninstall (${reason(e)}); nothing was uninstalled`)
|
||||
}
|
||||
return await this.#upstream.write(server, API[action], { name })
|
||||
case 'config-set': {
|
||||
if (config === undefined || config.length > MAX_CONFIG) throw new Error('the config is missing or larger than 1 MB')
|
||||
try {
|
||||
await this.saveHistory(server, name, 'before-apply')
|
||||
} catch (e) {
|
||||
throw new Error(`the old config could not be saved first (${reason(e)}); nothing was changed`)
|
||||
}
|
||||
return await this.#upstream.write(server, API[action], {}, new URLSearchParams({ name, config }))
|
||||
}
|
||||
case 'install-file':
|
||||
case 'update-file': {
|
||||
if (file === undefined) throw new Error('no zip file')
|
||||
|
||||
+11
-1
@@ -33,12 +33,22 @@ export const ROUTES: Record<string, Route> = {
|
||||
'hub/audit/list': read('audit.read'),
|
||||
'hub/apps/matrix': read(),
|
||||
'apps/listStoreApps': read(),
|
||||
// Writes. They go to a server's ADMIN token only; see apps-write.ts. apps/config/set waits for a later stage.
|
||||
// Writes. They go to a server's ADMIN token only; see apps-write.ts.
|
||||
'apps/downloadAndInstall': writeOne('apps.write'),
|
||||
'apps/downloadAndUpdate': writeOne('apps.write'),
|
||||
'apps/uninstall': writeOne('apps.write'),
|
||||
'apps/install': writeOne('apps.write'),
|
||||
'apps/update': writeOne('apps.write'),
|
||||
// Config sync (apps-config.ts). Compare shows live configs to anyone who can read; the rest needs apps.write.
|
||||
'apps/config/get': read(),
|
||||
'apps/config/set': writeOne('apps.write'),
|
||||
'hub/apps/config/compare': read(),
|
||||
'hub/apps/config/preview': read('apps.write'),
|
||||
'hub/apps/overrides/current': read('apps.write'),
|
||||
'hub/apps/baseline/import': writeAll('apps.write'),
|
||||
'hub/apps/baseline/save': writeAll('apps.write'),
|
||||
'hub/apps/overrides/save': writeAll('apps.write'),
|
||||
'hub/apps/config/apply': writeAll('apps.write'),
|
||||
'hub/apps/run': writeAll('apps.write'),
|
||||
'hub/apps/upload': writeAll('apps.write'),
|
||||
}
|
||||
|
||||
+93
-5
@@ -24,6 +24,7 @@ import { routeFor } from './routes.ts'
|
||||
import { buildMatrix } from './apps-matrix.ts'
|
||||
import { AppsWriter, actionForPath, type FileAction, type TargetResult } from './apps-write.ts'
|
||||
import { TooLarge, parseMultipart, readLimited, type UploadedFile } from './multipart.ts'
|
||||
import { AppsConfig, UserError } from './apps-config.ts'
|
||||
import {
|
||||
mergeQueryLogs,
|
||||
mergeSettled,
|
||||
@@ -38,6 +39,8 @@ import {
|
||||
type Json = Record<string, unknown>
|
||||
|
||||
const MAX_BODY = 64 * 1024
|
||||
/* The config endpoints carry config texts (up to 1 MB). */
|
||||
const MAX_CONFIG_BODY = 2 * 1024 * 1024
|
||||
|
||||
const TYPES: Record<string, string> = {
|
||||
'.html': 'text/html; charset=utf-8',
|
||||
@@ -72,17 +75,17 @@ function send(res: http.ServerResponse, code: number, body: unknown): void {
|
||||
|
||||
const fail = (errorMessage: string): Json => ({ status: 'error', errorMessage })
|
||||
|
||||
async function readForm(req: http.IncomingMessage): Promise<URLSearchParams> {
|
||||
async function readForm(req: http.IncomingMessage, limit: number = MAX_BODY): Promise<URLSearchParams> {
|
||||
if (req.method !== 'POST') return new URLSearchParams()
|
||||
const chunks: Buffer[] = []
|
||||
let size = 0
|
||||
for await (const chunk of req) {
|
||||
size += (chunk as Buffer).length
|
||||
if (size > MAX_BODY) break
|
||||
if (size > limit) break
|
||||
chunks.push(chunk as Buffer)
|
||||
}
|
||||
const type = String(req.headers['content-type'] ?? '')
|
||||
if (size > MAX_BODY) return new URLSearchParams()
|
||||
if (size > limit) return new URLSearchParams()
|
||||
const text = Buffer.concat(chunks).toString('utf8')
|
||||
if (type.includes('application/x-www-form-urlencoded')) return new URLSearchParams(text)
|
||||
if (type.includes('application/json')) {
|
||||
@@ -93,6 +96,7 @@ async function readForm(req: http.IncomingMessage): Promise<URLSearchParams> {
|
||||
for (const [k, v] of Object.entries(data)) {
|
||||
if (Array.isArray(v)) out.set(k, v.map(String).join(','))
|
||||
else if (typeof v === 'string' || typeof v === 'number' || typeof v === 'boolean') out.set(k, String(v))
|
||||
else if (v !== null && typeof v === 'object') out.set(k, JSON.stringify(v))
|
||||
}
|
||||
} catch {
|
||||
// Not JSON: no parameters.
|
||||
@@ -131,6 +135,7 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
|
||||
const limiter = new LoginLimiter()
|
||||
const audit = new Audit(config.dataDir)
|
||||
const writer = new AppsWriter(config, upstream)
|
||||
const appsConfig = new AppsConfig(config, upstream)
|
||||
const byId = new Map(config.servers.map((s) => [s.id, s]))
|
||||
|
||||
/* Ask every server; the ones that fail become warnings (see mergeSettled). */
|
||||
@@ -243,6 +248,74 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
|
||||
return file
|
||||
}
|
||||
|
||||
/* A value from a JSON-encoded parameter, or the fallback. */
|
||||
function jsonParam(value: string | undefined, fallback: unknown): unknown {
|
||||
if (value === undefined || value === '') return fallback
|
||||
try {
|
||||
return JSON.parse(value)
|
||||
} catch {
|
||||
throw new UserError('A parameter is not valid JSON.')
|
||||
}
|
||||
}
|
||||
|
||||
function userFailure(res: http.ServerResponse, e: unknown): void {
|
||||
if (e instanceof UserError) send(res, 400, fail(e.message))
|
||||
else throw e
|
||||
}
|
||||
|
||||
/* compare, preview and the live local values: no change is made. */
|
||||
async function configRead(res: http.ServerResponse, apiPath: string, params: Params, who: Who, ip: string): Promise<void> {
|
||||
const name = params.name ?? ''
|
||||
try {
|
||||
if (name === '') throw new UserError('An app name is required.')
|
||||
if (apiPath === 'hub/apps/config/compare') send(res, 200, ok(await appsConfig.compare(name)))
|
||||
else if (apiPath === 'hub/apps/config/preview') {
|
||||
const preview = await appsConfig.preview(name, csv(params.targets))
|
||||
audit.log({ user: who.user, role: who.role, ip, action: 'apps.config-preview', target: name, outcome: 'ok', detail: csv(params.targets).join(',') })
|
||||
send(res, 200, ok(preview))
|
||||
} else if (apiPath === 'hub/apps/overrides/current') {
|
||||
send(res, 200, ok({ values: await appsConfig.currentLocalValues(name, csv(params.paths)) }))
|
||||
} else send(res, 403, fail('The hub is read-only.'))
|
||||
} catch (e) {
|
||||
userFailure(res, e)
|
||||
}
|
||||
}
|
||||
|
||||
/* baseline import and save, local fields save, apply. */
|
||||
async function configWrite(res: http.ServerResponse, apiPath: string, params: Params, who: Who, ip: string): Promise<void> {
|
||||
const name = params.name ?? ''
|
||||
const base = { user: who.user, role: who.role, ip, target: name }
|
||||
const action = apiPath.replace('hub/apps/', 'apps.').replace(/\//g, '-')
|
||||
try {
|
||||
if (name === '' || name.length > 200) throw new UserError('An app name is required.')
|
||||
if (apiPath === 'hub/apps/baseline/import') {
|
||||
const done = await appsConfig.importBaseline(name, params.server ?? '', who.user)
|
||||
audit.log({ ...base, action, server: params.server, outcome: 'ok', detail: done.detail })
|
||||
send(res, 200, ok({ baseline: appsConfig.getBaseline(name) }))
|
||||
} else if (apiPath === 'hub/apps/baseline/save') {
|
||||
const done = appsConfig.saveBaseline(name, params.text ?? '', who.user)
|
||||
audit.log({ ...base, action, outcome: 'ok', detail: done.detail })
|
||||
send(res, 200, ok({ baseline: appsConfig.getBaseline(name) }))
|
||||
} else if (apiPath === 'hub/apps/overrides/save') {
|
||||
const done = appsConfig.saveOverrides(name, csv(params.paths), jsonParam(params.values, {}))
|
||||
audit.log({ ...base, action, outcome: 'ok', detail: done.detail })
|
||||
send(res, 200, ok({ overrides: appsConfig.getOverrides(name) }))
|
||||
} else if (apiPath === 'hub/apps/config/apply') {
|
||||
const expect = jsonParam(params.expect, {}) as Record<string, string>
|
||||
const done = await appsConfig.apply(name, csv(params.targets), expect)
|
||||
for (const r of done.results) {
|
||||
audit.log({ ...base, action: 'apps.config-apply', server: r.server, outcome: r.ok ? 'ok' : 'failed', detail: [r.message, done.details[r.server]].filter(Boolean).join(' | ') })
|
||||
}
|
||||
send(res, 200, ok({ results: done.results }))
|
||||
} else send(res, 403, fail('The hub is read-only.'))
|
||||
} catch (e) {
|
||||
if (e instanceof UserError) {
|
||||
audit.log({ ...base, action, outcome: 'refused', detail: apiPath.endsWith('baseline/save') ? 'invalid baseline' : e.message.slice(0, 120) })
|
||||
}
|
||||
userFailure(res, e)
|
||||
}
|
||||
}
|
||||
|
||||
/* The write endpoints. The role has already been checked against the route's capability. */
|
||||
async function write(req: http.IncomingMessage, res: http.ServerResponse, apiPath: string, kind: string, params: Params, who: Who, ip: string): Promise<void> {
|
||||
if (kind === 'write-all') {
|
||||
@@ -250,6 +323,10 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
|
||||
send(res, 405, fail('Use POST.'))
|
||||
return
|
||||
}
|
||||
if (apiPath.startsWith('hub/apps/baseline/') || apiPath.startsWith('hub/apps/overrides/') || apiPath === 'hub/apps/config/apply') {
|
||||
await configWrite(res, apiPath, params, who, ip)
|
||||
return
|
||||
}
|
||||
if (apiPath === 'hub/apps/run') {
|
||||
const action = params.action
|
||||
if (action !== 'install-store' && action !== 'update-store' && action !== 'uninstall') {
|
||||
@@ -324,7 +401,7 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
|
||||
}
|
||||
url = fromStore ?? undefined
|
||||
}
|
||||
response = await writer.perform(server, action, name, url, file)
|
||||
response = await writer.perform(server, action, name, url, file, params.config)
|
||||
result = { server: server.id, ok: true }
|
||||
} catch (e) {
|
||||
result = { server: server.id, ok: false, message: e instanceof Error ? e.message : 'failed' }
|
||||
@@ -335,7 +412,10 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
|
||||
|
||||
async function api(req: http.IncomingMessage, res: http.ServerResponse, apiPath: string, url: URL): Promise<void> {
|
||||
// A multipart body (an app zip) is read later, once the caller is known to be allowed to send it.
|
||||
const form = isMultipart(req) ? new URLSearchParams() : await readForm(req)
|
||||
// A config body may be large, but only for a signed-in caller (token in the header or the query).
|
||||
const large = (apiPath.startsWith('hub/apps/') || apiPath === 'apps/config/set')
|
||||
&& sessions.touch(bearer(req) ?? url.searchParams.get('token')) !== null
|
||||
const form = isMultipart(req) ? new URLSearchParams() : await readForm(req, large ? MAX_CONFIG_BODY : MAX_BODY)
|
||||
const params: Params = {}
|
||||
for (const [k, v] of url.searchParams) params[k] = v
|
||||
for (const [k, v] of form) params[k] = v
|
||||
@@ -409,6 +489,14 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
|
||||
await write(req, res, apiPath, route.kind, params, who, ip)
|
||||
return
|
||||
}
|
||||
if (apiPath.startsWith('hub/apps/config/') || apiPath === 'hub/apps/overrides/current') {
|
||||
await configRead(res, apiPath, params, who, ip)
|
||||
return
|
||||
}
|
||||
if (apiPath === 'apps/config/get' && !byId.has(params.node ?? '')) {
|
||||
send(res, 400, fail('Choose one server: a config is read from a single server.'))
|
||||
return
|
||||
}
|
||||
if (apiPath === 'hub/apps/matrix') {
|
||||
try {
|
||||
const { matrix, warnings } = await buildMatrix(config, upstream)
|
||||
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
/*
|
||||
What the hub keeps per app under `${HUB_DATA}/apps/<app>/`:
|
||||
|
||||
baseline.json the desired config, as TEXT (some configs are not JSON)
|
||||
overrides.json the fields that stay local to each server, and their values
|
||||
history/ the config a server had before the hub changed it (50 files kept)
|
||||
|
||||
Every write is a temp file and a rename, mode 0600: these files hold configs.
|
||||
*/
|
||||
import { createHash } from 'node:crypto'
|
||||
import { existsSync, mkdirSync, readFileSync, readdirSync, renameSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import path from 'node:path'
|
||||
|
||||
export const KEEP_HISTORY = 50
|
||||
|
||||
/* A folder name for an app: letters, digits and . _ - only, with a short hash when anything had to change. */
|
||||
export function safeName(name: string): string {
|
||||
const cleaned = name.replace(/[^A-Za-z0-9._-]/g, '_').replace(/\.{2,}/g, '_').replace(/^\.+/, '_').slice(0, 80)
|
||||
if (cleaned === name && cleaned !== '') return cleaned
|
||||
return `${cleaned === '' ? '_' : cleaned}-${createHash('sha256').update(name).digest('hex').slice(0, 6)}`
|
||||
}
|
||||
|
||||
export const appDir = (dataDir: string, app: string): string => path.join(dataDir, 'apps', safeName(app))
|
||||
|
||||
export function writeAtomic(file: string, text: string): void {
|
||||
mkdirSync(path.dirname(file), { recursive: true })
|
||||
const temp = `${file}.${process.pid}.tmp`
|
||||
try {
|
||||
writeFileSync(temp, text, { mode: 0o600 })
|
||||
renameSync(temp, file)
|
||||
} catch (e) {
|
||||
rmSync(temp, { force: true })
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
export interface Baseline {
|
||||
app: string
|
||||
text: string
|
||||
importedFrom?: string
|
||||
updatedAt: string
|
||||
updatedBy: string
|
||||
}
|
||||
|
||||
export interface Overrides {
|
||||
/** Dot paths (`a.b.0.c`) of the fields that are not synchronised. */
|
||||
paths: string[]
|
||||
/** server -> path -> value */
|
||||
values: Record<string, Record<string, unknown>>
|
||||
}
|
||||
|
||||
export const EMPTY_OVERRIDES: Overrides = { paths: [], values: {} }
|
||||
|
||||
function readJson(file: string): unknown {
|
||||
if (!existsSync(file)) return null
|
||||
try {
|
||||
return JSON.parse(readFileSync(file, 'utf8'))
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export function readBaseline(dataDir: string, app: string): Baseline | null {
|
||||
const data = readJson(path.join(appDir(dataDir, app), 'baseline.json')) as Partial<Baseline> | null
|
||||
if (data === null || typeof data.text !== 'string') return null
|
||||
return {
|
||||
app,
|
||||
text: data.text,
|
||||
...(typeof data.importedFrom === 'string' ? { importedFrom: data.importedFrom } : {}),
|
||||
updatedAt: String(data.updatedAt ?? ''),
|
||||
updatedBy: String(data.updatedBy ?? ''),
|
||||
}
|
||||
}
|
||||
|
||||
export function writeBaseline(dataDir: string, baseline: Baseline): void {
|
||||
writeAtomic(path.join(appDir(dataDir, baseline.app), 'baseline.json'), JSON.stringify(baseline, null, 2) + '\n')
|
||||
}
|
||||
|
||||
export function readOverrides(dataDir: string, app: string): Overrides {
|
||||
const data = readJson(path.join(appDir(dataDir, app), 'overrides.json')) as Partial<Overrides> | null
|
||||
if (data === null || !Array.isArray(data.paths)) return { paths: [], values: {} }
|
||||
const values = data.values !== null && typeof data.values === 'object' ? data.values : {}
|
||||
return { paths: data.paths.filter((p): p is string => typeof p === 'string'), values }
|
||||
}
|
||||
|
||||
export function writeOverrides(dataDir: string, app: string, overrides: Overrides): void {
|
||||
writeAtomic(path.join(appDir(dataDir, app), 'overrides.json'), JSON.stringify(overrides, null, 2) + '\n')
|
||||
}
|
||||
|
||||
/*
|
||||
Saves a server's config as it was before the hub changed it. The name starts with the time, so
|
||||
sorting the names sorts the files by age; the oldest beyond KEEP_HISTORY are deleted.
|
||||
*/
|
||||
export function writeHistory(
|
||||
dataDir: string,
|
||||
app: string,
|
||||
server: string,
|
||||
text: string,
|
||||
kind: 'before-uninstall' | 'before-apply',
|
||||
now: number,
|
||||
): string {
|
||||
const dir = path.join(appDir(dataDir, app), 'history')
|
||||
const stamp = new Date(now).toISOString().replace(/:/g, '-')
|
||||
const file = path.join(dir, `${stamp}__${safeName(server)}__${kind}.txt`)
|
||||
writeAtomic(file, text)
|
||||
const files = readdirSync(dir).filter((f) => f.endsWith('.txt')).sort()
|
||||
for (const old of files.slice(0, Math.max(0, files.length - KEEP_HISTORY))) rmSync(path.join(dir, old), { force: true })
|
||||
return file
|
||||
}
|
||||
+1
-1
@@ -63,7 +63,7 @@ export class Upstream {
|
||||
the outcome, what the hub remembers of this server is dropped afterwards, so the next read
|
||||
shows the state the write left. `body` is a multipart upload (the app zip).
|
||||
*/
|
||||
async write(server: HubServer, path: string, params: Params = {}, body?: FormData): Promise<unknown> {
|
||||
async write(server: HubServer, path: string, params: Params = {}, body?: FormData | URLSearchParams): Promise<unknown> {
|
||||
if (server.adminToken === undefined) throw new UpstreamError('rejected', 'no admin token configured')
|
||||
try {
|
||||
const init: RequestInit = body === undefined ? { method: 'POST' } : { method: 'POST', body }
|
||||
|
||||
Reference in new issue
Block a user