dc803bcb34e8f5f547856349a9d50d054267bb89
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
53ce80c9ff |
fix(pipeline): harden upstream resilience and production shutdown
DoH через DoWithRetry; CDN preview через UpstreamHTTPDo; частичный fail CDN (EVOBGP_CDN_PARTIAL_OK); безопасный доступ к Job.Meta; drain jobs при SIGTERM; ValidateProductionEnforce при EVOBGP_PRODUCTION=1. Co-authored-by: Cursor <[email protected]> |
||
|
|
8c97445f7e |
fix(jobs): make deploy coalescing deterministic via inflight counter
CI / changes (push) Successful in 12s
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / commitlint (push) Has been skipped
CI / go (push) Successful in 1m11s
CI / bird2 (push) Successful in 16s
CI / release (push) Successful in 4m27s
TestParallelModuleRefresh_CoalescesDeployApply падал на CI под -race (want exactly one deploy_apply job, got 2). Локально тест проходил стабильно (100/500 итераций с -cpu), но узкая гонка проявлялась при замедлении под race-детектором. Корень: коалесцирование решало «делать ли deploy» через CountOtherActiveRefresh, который опрашивал статусы job-ов (queued/ running). Статусы меняются асинхронно относительно tenantRefreshMu, поэтому в редких таймингах оба параллельных refresh могли решить, что другой уже не активен, и каждый породил свой deploy_apply. Решение — детерминированный inflight-счётчик refresh-kind job-ов в Registry (inflightRefresh map[string]int), управляемый под r.mu: - инкремент в Enqueue при создании нового refresh-kind job-а; - декремент + проверка «последний ли я» в finishModuleRefreshSuccess через новый метод finalizeRefreshCoalesce (под tenantRefreshMu). Последний refresh (счётчик <= 1) делает render + deploy_apply; все остальные defer-ят. Решение больше не зависит от опроса статусов и таймингов ingest. Чтобы счётчик не утёк на error/cancel путях (где refresh не доходит до finishModuleRefreshSuccess), обработка module_refresh и tenant_refresh вынесена в runModuleRefresh / runTenantRefresh с defer-обёрткой, которая гарантированно освобождает слот, если finishModuleRefreshSuccess не отработал. CountOtherActiveRefresh / CountOtherActiveModuleRefresh оставлены как публичные методы (могут использоваться в мониторинге); из продакшн-логики коалесцирования убраны. Проверки: go build, go vet, go test ./internal/... -count=1 — exit 0. Стресс-тест коалесцирования: 200 итераций с -cpu=4 — стабильно. Co-authored-by: Cursor <[email protected]> |
||
|
|
e65cf0d958 |
feat(jobs): persist job lifecycle to PostgreSQL job_audit
UpsertQueued/Running/MarkTerminal через SetPersistHooks; исправлен deadlock fireEnqueued под Registry mutex. Co-authored-by: Cursor <[email protected]> |
||
|
|
8ebce28e34 |
perf: wire job_audit terminal persistence hook
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Failing after 35s
CI / go (push) Failing after 30s
CI / bird2 (push) Has been skipped
CI / release (push) Has been skipped
- TerminalHook в Registry для записи статуса job в PostgreSQL job_audit - Подключение через BootstrapWorkers при наличии pool Co-authored-by: Cursor <[email protected]> |
||
|
|
dd7d43c2c2 |
perf: incremental render scale, SQL pagination and observability
- BIRD filter chunking (500 CIDR); bounded job worker pool
- ListModulesPage SQL push-down; revision diff limit; batch revision prune
- DB pool tuning; Prometheus pipeline/job metrics
- Coalesce module_refresh via idempotency; GET /jobs/{id}/report
- JobAuditWriter foundation for job_audit persistence
Co-authored-by: Cursor <[email protected]>
|
||
|
|
7a2b015b12 |
feat: add tenant refresh functionality to HTTP API and job processing
CI / changes (push) Successful in 11s
CI / openapi (push) Has been skipped
CI / go (push) Successful in 59s
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, , evobgp-web) (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, evobgp-all, evobgp-web-all) (push) Has been skipped
CI / bird2 (push) Successful in 43s
CI / docker-go (evobgp-all, 1, deploy/docker/gobinary/Dockerfile, , evobgp-all) (push) Has been skipped
CI / docker-go (evobgp-api, 1, deploy/docker/gobinary/Dockerfile, , evobgp-api) (push) Has been skipped
CI / docker-go (evobgp-deploy, 0, deploy/docker/gobinary/Dockerfile, , evobgp-deploy) (push) Has been skipped
CI / docker-go (evobgp-ingest, 0, deploy/docker/gobinary/Dockerfile, , evobgp-ingest) (push) Has been skipped
CI / docker-go (evobgp-node, 0, deploy/docker/gobinary/Dockerfile, , evobgp-node) (push) Has been skipped
CI / docker-go (evobgp-render, 0, deploy/docker/gobinary/Dockerfile, , evobgp-render) (push) Has been skipped
CI / docker-go (evobgp-scheduler, 0, deploy/docker/gobinary/Dockerfile, , evobgp-scheduler) (push) Has been skipped
CI / docker-go-prime (push) Successful in 55s
CI / docker-go (deploy/docker/evobgp-agent/Dockerfile, , evobgp-agent) (push) Has been cancelled
Implemented a new endpoint for tenant refresh in the HTTP API, allowing for the refresh of modules associated with a tenant. Enhanced job processing to handle tenant refresh jobs, including logic for managing module IDs and job status updates. Updated the scheduler to enqueue tenant refresh jobs based on module due dates, improving the overall efficiency of module management. Additionally, introduced caching for ASN prefix data to optimize performance during refresh operations. |
||
|
|
f6b94a44d0 |
refactor: enhance bird metrics polling with context support
CI / changes (push) Successful in 8s
CI / openapi (push) Has been skipped
CI / go (push) Successful in 40s
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, , evobgp-web) (push) Has been skipped
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, evobgp-all, evobgp-web-all) (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 15s
CI / docker-go-prime (push) Successful in 24s
CI / docker-go (deploy/docker/evobgp-agent/Dockerfile, , evobgp-agent) (push) Successful in 59s
CI / docker-go (evobgp-all, 1, deploy/docker/gobinary/Dockerfile, , evobgp-all) (push) Successful in 2m18s
CI / docker-go (evobgp-api, 1, deploy/docker/gobinary/Dockerfile, , evobgp-api) (push) Successful in 1m23s
CI / docker-go (evobgp-deploy, 0, deploy/docker/gobinary/Dockerfile, , evobgp-deploy) (push) Successful in 1m20s
CI / docker-go (evobgp-ingest, 0, deploy/docker/gobinary/Dockerfile, , evobgp-ingest) (push) Successful in 1m24s
CI / docker-go (evobgp-node, 0, deploy/docker/gobinary/Dockerfile, , evobgp-node) (push) Successful in 1m9s
CI / docker-go (evobgp-render, 0, deploy/docker/gobinary/Dockerfile, , evobgp-render) (push) Successful in 1m22s
CI / docker-go (evobgp-scheduler, 0, deploy/docker/gobinary/Dockerfile, , evobgp-scheduler) (push) Successful in 1m24s
Updated the `startBirdMetricsPoller` function to accept a context parameter, allowing for better control over the polling lifecycle. This change was applied in both `evobgp-all` and `evobgp-api` main files. Additionally, modified the `StartBirdProtocolsPoller` function to handle context cancellation, ensuring graceful shutdown of the polling routine. Introduced a new service in the Docker Compose configuration for logging runtime service outputs, improving observability during deployment. |
||
|
|
d83efe24ef |
refactor: optimize job listing and counting logic for concurrency safety. Separate tenant filtering and status checks to reduce lock contention, improving performance and ensuring accurate job status retrieval.
CI / changes (push) Successful in 14s
CI / openapi (push) Has been skipped
CI / go (push) Successful in 1m12s
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, , evobgp-web) (push) Has been skipped
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, evobgp-all, evobgp-web-all) (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 20s
CI / docker-go-prime (push) Successful in 29s
CI / docker-go (deploy/docker/evobgp-agent/Dockerfile, , evobgp-agent) (push) Successful in 1m19s
CI / docker-go (evobgp-all, 1, deploy/docker/gobinary/Dockerfile, , evobgp-all) (push) Successful in 2m54s
CI / docker-go (evobgp-api, 1, deploy/docker/gobinary/Dockerfile, , evobgp-api) (push) Successful in 1m49s
CI / docker-go (evobgp-deploy, 0, deploy/docker/gobinary/Dockerfile, , evobgp-deploy) (push) Successful in 1m47s
CI / docker-go (evobgp-ingest, 0, deploy/docker/gobinary/Dockerfile, , evobgp-ingest) (push) Successful in 1m45s
CI / docker-go (evobgp-node, 0, deploy/docker/gobinary/Dockerfile, , evobgp-node) (push) Successful in 1m14s
CI / docker-go (evobgp-render, 0, deploy/docker/gobinary/Dockerfile, , evobgp-render) (push) Successful in 1m30s
CI / docker-go (evobgp-scheduler, 0, deploy/docker/gobinary/Dockerfile, , evobgp-scheduler) (push) Successful in 1m26s
|
||
|
|
149fb0a18c |
feat: implement module refresh job management with concurrency control. Add CountOtherActiveModuleRefresh method to track active jobs and enhance finishModuleRefreshSuccess to defer deploy_apply when parallel refreshes are detected, improving job processing efficiency.
CI / changes (push) Successful in 6s
CI / openapi (push) Has been skipped
CI / go (push) Failing after 42s
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, , evobgp-web) (push) Has been skipped
CI / docker-web (deploy/docker/evobgp-web/Dockerfile, evobgp-all, evobgp-web-all) (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / docker-go-prime (push) Has been skipped
CI / docker-go (deploy/docker/evobgp-agent/Dockerfile, , evobgp-agent) (push) Has been skipped
CI / docker-go (evobgp-all, 1, deploy/docker/gobinary/Dockerfile, , evobgp-all) (push) Has been skipped
CI / docker-go (evobgp-api, 1, deploy/docker/gobinary/Dockerfile, , evobgp-api) (push) Has been skipped
CI / docker-go (evobgp-deploy, 0, deploy/docker/gobinary/Dockerfile, , evobgp-deploy) (push) Has been skipped
CI / docker-go (evobgp-ingest, 0, deploy/docker/gobinary/Dockerfile, , evobgp-ingest) (push) Has been skipped
CI / docker-go (evobgp-node, 0, deploy/docker/gobinary/Dockerfile, , evobgp-node) (push) Has been skipped
CI / docker-go (evobgp-render, 0, deploy/docker/gobinary/Dockerfile, , evobgp-render) (push) Has been skipped
CI / docker-go (evobgp-scheduler, 0, deploy/docker/gobinary/Dockerfile, , evobgp-scheduler) (push) Has been skipped
|
||
|
|
acc0608432 |
docs: update architecture and API documentation to clarify job processing, in-memory registry usage, and future broker integration plans. Enhance OpenAPI specifications for enrollment responses and refine quickstart instructions for Docker setups.
CI / changes (push) Successful in 4s
CI / openapi (push) Successful in 24s
CI / go (push) Successful in 22s
CI / bird2 (push) Successful in 13s
CI / docker-images (deploy/docker/bird2/Dockerfile, evobgp-bird2) (push) Successful in 38s
CI / docker-images (deploy/docker/evobgp-agent/Dockerfile, evobgp-agent) (push) Successful in 1m8s
CI / docker-images (deploy/docker/evobgp-web/Dockerfile, evobgp-web) (push) Successful in 49s
CI / docker-images (evobgp-all, 1, deploy/docker/gobinary/Dockerfile, evobgp-all) (push) Successful in 1m30s
CI / docker-images (evobgp-api, 1, deploy/docker/gobinary/Dockerfile, evobgp-api) (push) Successful in 1m31s
CI / docker-images (evobgp-ingest, 0, deploy/docker/gobinary/Dockerfile, evobgp-ingest) (push) Has been cancelled
CI / docker-images (evobgp-deploy, 0, deploy/docker/gobinary/Dockerfile, evobgp-deploy) (push) Has been cancelled
CI / docker-images (evobgp-node, 0, deploy/docker/gobinary/Dockerfile, evobgp-node) (push) Has been cancelled
CI / docker-images (evobgp-render, 0, deploy/docker/gobinary/Dockerfile, evobgp-render) (push) Has been cancelled
CI / docker-images (evobgp-scheduler, 0, deploy/docker/gobinary/Dockerfile, evobgp-scheduler) (push) Has been cancelled
|
||
|
|
bf52b21150 | feat: enhance evobgp with new command-line tools for bundle management, including pull, verify, and apply functionalities. Update go.mod to include necessary dependencies and complete todos in architecture plan for improved observability and deployment practices. |