Denozordec
7699e8b809
3.4 KiB
Statistics maintenance resource correction
The production report showed sustained CPU and substantial disk IO after the update. Inspection found that startup immediately drained every pending legacy statistics row: each sample opened a transaction, changed its pending index and rewrote additive rollup buckets. setImmediate yielded to requests but did not throttle this work. Existing logs do not establish that this is the sole production cause; no production CPU/IO profile was obtained.
The correction processes durable, atomic batches and merges observations sharing an agent/hour/day bucket before writing. Default maintenance processes at most 1000 samples or approximately one second per invocation, with up to 50 samples per transaction, a cooperative 10ms computation check and real 25ms inter-batch pauses. The computation check is not a hard deadline for synchronous SQLite writes. Cleanup performs at most two deletion batches per tier per invocation.
The application waits 30 seconds before its first maintenance tick and resumes every 30 seconds with one shared in-flight promise. Backfill/retention also reject overlapping passes for the same DB object. Shutdown aborts waits and awaits the active pass before DB closure. Pending rows remain durable between ticks. Processed raw predecessors are retained until the entire backfill backlog drains, so resumed legacy intervals retain their counter baseline; raw deletion still requires processed status. Marks and rollup writes commit or roll back together, preventing replay double-counting.
Local before/after observation
The same Windows Node/SQLite file-backed fixture seeded 4000 legacy samples across eight agents, eight days old. After seeding it truncated the WAL and disabled automatic checkpointing, making generated WAL bytes observable. Each profile drained all pending work, checked exact hourly/daily dropped and accepted totals, and verified that a second pass processed zero rows. No wall-time assertion or production capacity claim is made.
| Observation | Previous implementation | Corrected implementation |
|---|---|---|
| Wall time | 7341.96ms | 3877.81ms |
| Process CPU | 5641ms | 515ms |
| Generated WAL bytes | 79561352 | 3860472 |
| SQLite row changes | 12000 | 5380 |
The corrected wall time includes real inter-batch pauses. These were isolated local observations, not production measurements; WAL auto-checkpointing was deliberately disabled for both. The profile drains successive bounded passes immediately for comparison, while the actual application spaces passes 30 seconds apart. Its maximum sample budget therefore allows at most about 33 legacy samples/second averaged across ticks; actual progress depends on the one-second budget and host load. Progress logs report processed/deleted counts and whether more pending history remains. A large backlog is resumed rather than silently discarded.
Focused regressions cover predecessor preservation across partial backfill/retention/resume, transaction rollback, exact-once retries, abort/resume, overlapping passes and one-write bucket coalescing with preserved flags/counters. Root's complete API suite passed 285 tests and shared/DB/API production builds passed. A full-suite concurrent profile also retained the reduction (3889312 WAL bytes); its CPU/timing is not substituted for the isolated comparison above. Astra review found no confirmed P1/P2 issue. Production CPU, IO and backlog progress still need observation after rollout.