29 Commits

Include renames

Author	SHA1	Message	Date
Denozordec	3fd05ff833	fix(tests): improve concurrency handling in worker tests and enhance module cloning ... - Added environment variable `EVOBGP_JOB_MAX_CONCURRENT` to control job concurrency in tests. - Modified worker test to ensure proper synchronization of job processing by holding workers until both jobs are enqueued. - Updated memory store methods to return cloned module instances, preventing unintended mutations of original modules during operations.	2026-08-18 19:58:39 +07:00
DenozordecandCursor	8fe74c1d3b	feat(jobs): add durable PG queue reclaim, slog, and richer metrics ... JSON slog в ключевых пакетах; Prometheus path_group, job_audit_depth, upstream breaker; job_audit ClaimQueued/ReclaimStaleRunning + Adopt loop для HA после рестарта. Co-authored-by: Cursor <[email protected]>	2026-07-31 12:26:18 +07:00
DenozordecandCursor	53ce80c9ff	fix(pipeline): harden upstream resilience and production shutdown ... DoH через DoWithRetry; CDN preview через UpstreamHTTPDo; частичный fail CDN (EVOBGP_CDN_PARTIAL_OK); безопасный доступ к Job.Meta; drain jobs при SIGTERM; ValidateProductionEnforce при EVOBGP_PRODUCTION=1. Co-authored-by: Cursor <[email protected]>	2026-07-31 12:11:52 +07:00
DenozordecandCursor	8c97445f7e	fix(jobs): make deploy coalescing deterministic via inflight counter ... TestParallelModuleRefresh_CoalescesDeployApply падал на CI под -race (want exactly one deploy_apply job, got 2). Локально тест проходил стабильно (100/500 итераций с -cpu), но узкая гонка проявлялась при замедлении под race-детектором. Корень: коалесцирование решало «делать ли deploy» через CountOtherActiveRefresh, который опрашивал статусы job-ов (queued/ running). Статусы меняются асинхронно относительно tenantRefreshMu, поэтому в редких таймингах оба параллельных refresh могли решить, что другой уже не активен, и каждый породил свой deploy_apply. Решение — детерминированный inflight-счётчик refresh-kind job-ов в Registry (inflightRefresh map[string]int), управляемый под r.mu: - инкремент в Enqueue при создании нового refresh-kind job-а; - декремент + проверка «последний ли я» в finishModuleRefreshSuccess через новый метод finalizeRefreshCoalesce (под tenantRefreshMu). Последний refresh (счётчик <= 1) делает render + deploy_apply; все остальные defer-ят. Решение больше не зависит от опроса статусов и таймингов ingest. Чтобы счётчик не утёк на error/cancel путях (где refresh не доходит до finishModuleRefreshSuccess), обработка module_refresh и tenant_refresh вынесена в runModuleRefresh / runTenantRefresh с defer-обёрткой, которая гарантированно освобождает слот, если finishModuleRefreshSuccess не отработал. CountOtherActiveRefresh / CountOtherActiveModuleRefresh оставлены как публичные методы (могут использоваться в мониторинге); из продакшн-логики коалесцирования убраны. Проверки: go build, go vet, go test ./internal/... -count=1 — exit 0. Стресс-тест коалесцирования: 200 итераций с -cpu=4 — стабильно. Co-authored-by: Cursor <[email protected]>	2026-07-03 01:31:32 +07:00
DenozordecandCursor	cbf345b25f	refactor(maintenance): remove hardcoded retention and wire scheduler ... RunPeriodicMaintenance и RunCleanup удалены; scheduler политик в StartBackground; deprecated /postgres/cleanup принимает policy_id. Co-authored-by: Cursor <[email protected]>	2026-06-12 13:27:49 +07:00
DenozordecandCursor	f548d0671f	feat(api): add /v1/maintenance policies and run endpoints ... OpenAPI, httpapi CRUD/run/dry-run, job maintenance_policy_run и audit с policy_id. Co-authored-by: Cursor <[email protected]>	2026-06-12 13:23:10 +07:00
Denozordec	fad2bd3353	feat(db): implement PostgreSQL monitoring and maintenance features ... Added PostgreSQL monitoring and maintenance capabilities to the API, including new endpoints for instance-level metrics, maintenance operations, and job scheduling. Updated the HTTP API to support PostgreSQL monitoring routes and integrated a background scheduler for metrics collection. Enhanced the CLI with database commands for maintenance tasks. Updated documentation to reflect these changes.	2026-06-01 13:43:33 +07:00
DenozordecandCursor	e65cf0d958	feat(jobs): persist job lifecycle to PostgreSQL job_audit ... UpsertQueued/Running/MarkTerminal через SetPersistHooks; исправлен deadlock fireEnqueued под Registry mutex. Co-authored-by: Cursor <[email protected]>	2026-05-25 10:33:57 +07:00
DenozordecandCursor	782097420d	fix(httpclient): replace DefaultClient with timed clients and retry ... Пакет httpclient: timeout 45s, idle pool, DoWithRetry. Scheduler и nodecli используют retry; pipeline/asnresolve/jobs — httpclient.New вместо DefaultClient. Co-authored-by: Cursor <[email protected]>	2026-05-25 10:14:53 +07:00
Denozordec	e0a912a693	fix(worker): ensure job success is called before enqueueing deploy speakers ... - Moved the call to j.Succeed() to occur before enqueueing deploy speakers in the runPeerReconcile and finishModuleRefreshSuccess methods, ensuring proper job success handling.	2026-05-21 13:21:33 +07:00
Denozordec	2aecbf96fd	feat(remote-speakers): enhance remote speaker management and API integration ... - Added support for remote speaker configuration in the README and documentation. - Implemented a new endpoint for retrieving the bundle signing public key. - Updated the `evobgp-agent` to include a `serve` command for Panel→Node sync API. - Enhanced CI workflow to validate remote speaker compose files. - Introduced new fields in the API and UI for managing speaker metadata, including dispatch status and sync status. - Improved error handling and response formatting in speaker-related API endpoints. - Updated documentation to reflect changes in remote speaker functionality and usage guidelines.	2026-05-21 12:42:06 +07:00
DenozordecandCursor	8ebce28e34	perf: wire job_audit terminal persistence hook ... - TerminalHook в Registry для записи статуса job в PostgreSQL job_audit - Подключение через BootstrapWorkers при наличии pool Co-authored-by: Cursor <[email protected]>	2026-05-21 10:46:13 +07:00
DenozordecandCursor	dd7d43c2c2	perf: incremental render scale, SQL pagination and observability ... - BIRD filter chunking (500 CIDR); bounded job worker pool - ListModulesPage SQL push-down; revision diff limit; batch revision prune - DB pool tuning; Prometheus pipeline/job metrics - Coalesce module_refresh via idempotency; GET /jobs/{id}/report - JobAuditWriter foundation for job_audit persistence Co-authored-by: Cursor <[email protected]>	2026-05-21 10:45:46 +07:00
DenozordecandCursor	be3d73f374	perf: quick wins for pipeline, jobs, httpapi and web ops ... - CDN snapshot: batch merge после parallel fetch, без race на persist - ListRevisionPrefixes: лёгкая проверка revision вместо GetRevision - Jobs: timeout/cancel context для pipeline и deploy - HTTP server timeouts; кэш birdc для GET /peers - ListModules: batch DoH profiles одним запросом - Web: tab-scoped load на /operations, debounce job search, меньше over-fetch на dashboard Co-authored-by: Cursor <[email protected]>	2026-05-21 10:42:13 +07:00
Denozordec	7a2b015b12	feat: add tenant refresh functionality to HTTP API and job processing ... Implemented a new endpoint for tenant refresh in the HTTP API, allowing for the refresh of modules associated with a tenant. Enhanced job processing to handle tenant refresh jobs, including logic for managing module IDs and job status updates. Updated the scheduler to enqueue tenant refresh jobs based on module due dates, improving the overall efficiency of module management. Additionally, introduced caching for ASN prefix data to optimize performance during refresh operations.	2026-05-19 10:35:20 +07:00
Denozordec	6107c3eb87	refactor: enhance job status checks in worker tests ... Updated the worker test cases to improve the handling of job status checks. Added nil checks for job instances and refined the logic to verify job statuses directly from snapshots, ensuring more robust test coverage and preventing potential nil pointer dereferences.	2026-04-09 15:14:19 +07:00
Denozordec	3b17228ef2	feat: implement peer reconciliation job for BGP peers ... Added a new job type `peer_reconcile` to handle fast reconciliation of BGP peers without module ingestion. Updated API documentation to reflect the new job's functionality and its automatic application to speakers post-reconciliation. Enhanced the HTTP API to enqueue reconciliation jobs during peer creation, updates, and deletions, ensuring efficient state management for BGP configurations.	2026-04-09 15:07:21 +07:00
Denozordec	10e2415cfa	refactor: streamline module refresh process and enhance revision handling ... Updated the worker's job processing to utilize a new `RefreshModuleIngest` function, which simplifies the module refresh logic by separating ingestion from revision rendering. The `finishModuleRefreshSuccess` method now handles the aggregation of revision logs and enqueues deploy actions more efficiently. Additionally, refactored the `RefreshModule` function to maintain backward compatibility while improving clarity and functionality in the pipeline's revision management.	2026-04-08 13:42:53 +07:00
Denozordec	f6b94a44d0	refactor: enhance bird metrics polling with context support ... Updated the `startBirdMetricsPoller` function to accept a context parameter, allowing for better control over the polling lifecycle. This change was applied in both `evobgp-all` and `evobgp-api` main files. Additionally, modified the `StartBirdProtocolsPoller` function to handle context cancellation, ensuring graceful shutdown of the polling routine. Introduced a new service in the Docker Compose configuration for logging runtime service outputs, improving observability during deployment.	2026-04-08 12:29:07 +07:00
Denozordec	53fd527a6d	feat: enhance community label handling and improve UI localization across operations pages. Introduce community label mapping for better readability in job logs and update various components to reflect Russian translations, enhancing user experience and consistency in the interface.	2026-04-07 00:58:15 +07:00
Denozordec	d83efe24ef	refactor: optimize job listing and counting logic for concurrency safety. Separate tenant filtering and status checks to reduce lock contention, improving performance and ensuring accurate job status retrieval.	2026-04-07 00:03:46 +07:00
Denozordec	149fb0a18c	feat: implement module refresh job management with concurrency control. Add CountOtherActiveModuleRefresh method to track active jobs and enhance finishModuleRefreshSuccess to defer deploy_apply when parallel refreshes are detected, improving job processing efficiency.	2026-04-06 23:59:26 +07:00
Denozordec	c48cc4e2d4	feat: enhance job processing with detailed logging and UI improvements. Introduce revision log entries in the Worker to capture job metadata, including source and community details. Update the Svelte UI to display job logs with expandable details, improving user experience and job tracking capabilities.	2026-04-06 17:10:29 +07:00
Denozordec	42d956d109	feat: enhance worker functionality by adding job queuing for deploy_apply. Introduce a Registry field in the Worker struct and implement enqueueDeployAllSpeakers method to streamline job processing after refresh or rollback operations, improving deployment efficiency.	2026-04-06 16:32:23 +07:00
Denozordec	94a4c6acd4	feat: add BIRD status endpoint and integrate birdc checks into job processing. Enhance UI to display BIRD runtime status and job metadata, improving user feedback on BIRD operations.	2026-04-06 00:25:43 +07:00
Denozordec	acc0608432	docs: update architecture and API documentation to clarify job processing, in-memory registry usage, and future broker integration plans. Enhance OpenAPI specifications for enrollment responses and refine quickstart instructions for Docker setups.	2026-04-05 18:36:32 +07:00
Denozordec	b7968db4e0	refactor: update worker processes in EvoBGP to accept dependencies for shared store and job registry. Enhance scheduler, ingest, render, and deploy components to utilize a unified context and improve logging for drift detection. Update architecture documentation to reflect changes in process interactions and worker functionalities.	2026-04-05 17:42:07 +07:00
Denozordec	6a55f72ab3	feat: enhance EvoBGP with new command-line options for the evobgp-agent, including a watch command for periodic configuration updates. Update go.mod with additional dependencies and improve Docker Compose setup for new services, including NATS and various worker components.	2026-04-05 17:03:21 +07:00
Denozordec	bf52b21150	feat: enhance evobgp with new command-line tools for bundle management, including pull, verify, and apply functionalities. Update go.mod to include necessary dependencies and complete todos in architecture plan for improved observability and deployment practices.	2026-04-05 14:07:45 +07:00