denozord/strongswan-ext
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

github: Remove commit ID from cache keys and manually evict old entries

Browse Source 
Let's try this again :)  Since cache entries with the same key are not
updated/replaced and there is no option to do so, we manually delete the
previous entry for the current branch.

This reduces the cache storage for active branches, which can cause
caches of the master branch to get evicted, which in turn will slow down
builds of not only master but also new branches as they can't fall back
on those caches.

Permission has to be explicitly granted in order to delete the cache
entries when not using the legacy all-write tokens that are the default
for old repositories.

The continue-on-error option is set for the step that deletes the old
cache entry as it's expected that cache-hit will be true for a new feature
branch when restoring the cache from the master branch.  However, because
there won't be anything to delete for this branch yet, the command will
fail.  The --succeed-on-no-caches option of the command unfortunately
only works with --all.

For the Linux tests, several jobs use the same cache key.  So there is
a chance that two jobs try to store a new entry concurrently, which will
fail (it works if there was a cache hit and they are slightly off as
previous entries are first deleted).  To avoid that, we store the cache
only for one particular config.

Also made sure that the "openssl" test does not remove "openssl-3/4"
caches by adding a suffix to the former.

For alpine, the repository had to be set explicitly as gh wasn't able to
determine it (didn't detect the Git working dir).
This commit is contained in:
Tobias Brunner
2026-05-07 10:52:08 +02:00
parent 94443ebad1
commit 3a44941d2c
8 changed files with 194 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/linux.yml
+76 -20
View File
@@ -6,6 +6,9 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
actions: write
env:
# this test case does not actually test anything but tries to access system
# directories that might be inaccessible on build hosts
@@ -64,25 +67,44 @@ jobs:
MONOLITHIC: ${{ matrix.monolithic || 'no' }}
CC: ${{ matrix.compiler || 'gcc' }}
TEST: ${{ matrix.test }}
# as several jobs use the same key, make sure we only store the cache for
# one specific config in case there is a race
STORE_CACHE: >-
${{
!contains(fromJSON('["all", "default", "printf-builtin"]'),
matrix.test) ||
(matrix.leak-detective == 'no' && matrix.monolithic == 'no')
}}
steps:
- uses: actions/checkout@v6
- uses: actions/cache@v5
- uses: actions/cache/restore@v5
id: cache-restore
with:
path: ~/.cache/ccache
# with regards to ccache, monolithic builds don't differ from regular
# builds and, similarly, builds with leak-detective only differ in two
# files (LD itself and library.c); but different tests build different
# dependencies, so different caches are needed
key: ccache-ubuntu-latest-${{ env.CC }}-${{ matrix.test }}-${{ github.sha }}
key: ccache-ubuntu-latest-${{ env.CC }}-${{ matrix.test }}
restore-keys: |
ccache-ubuntu-latest-${{ env.CC }}-${{ matrix.test }}-
ccache-ubuntu-latest-${{ env.CC }}-
- run: |
sudo apt-get install -qq ccache
echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV
ccache -z
- uses: ./.github/actions/default
- run: ccache -s
- run: ccache -sv
# delete old cache entry as we currently can't update it any other way
- env:
GH_TOKEN: ${{ github.token }}
if: steps.cache-restore.outputs.cache-hit && fromJSON(env.STORE_CACHE)
continue-on-error: true
run: gh cache delete -r ${{ github.ref }} ${{ steps.cache-restore.outputs.cache-primary-key }}
- if: fromJSON(env.STORE_CACHE)
uses: actions/cache/save@v5
with:
path: ~/.cache/ccache
key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- if: ${{ success() && matrix.test == 'coverage' }}
uses: codecov/codecov-action@v4
with:
@@ -105,7 +127,7 @@ jobs:
strategy:
fail-fast: false
matrix:
test: [ botan, wolfssl, openssl, openssl-3, openssl-4, openssl-awslc, gcrypt ]
test: [ botan, wolfssl, openssl-sys, openssl-3, openssl-4, openssl-awslc, gcrypt ]
os: [ ubuntu-latest, ubuntu-22.04 ]
leak-detective: [ no, yes ]
exclude:
@@ -125,19 +147,19 @@ jobs:
CC: ${{ matrix.compiler || 'gcc' }}
TEST: ${{ matrix.test }}
ACTIVE_TRANSFORMS_REF: .github/active-transforms/${{ matrix.test }}
STORE_CACHE: ${{ !matrix.leak-detective || matrix.leak-detective == 'no' }}
steps:
- uses: actions/checkout@v6
- uses: actions/cache@v5
- uses: actions/cache/restore@v5
id: cache-restore
with:
# path is different on newer systems
path: |
~/.cache/ccache
~/.ccache
key: ccache-${{ matrix.os }}-${{ env.CC }}-${{ matrix.test }}-${{ github.sha }}
key: ccache-${{ matrix.os }}-${{ env.CC }}-${{ matrix.test }}
restore-keys: |
ccache-${{ matrix.os }}-${{ env.CC }}-${{ matrix.test }}-
ccache-${{ matrix.os }}-${{ env.CC }}-all-${{ github.sha }}
ccache-${{ matrix.os }}-${{ env.CC }}-all-
ccache-${{ matrix.os }}-${{ env.CC }}-all
ccache-${{ matrix.os }}-${{ env.CC }}-
- run: |
sudo apt-get install -qq ccache
@@ -154,7 +176,20 @@ jobs:
- name: Verify active transforms
run: |
test ! -f $ACTIVE_TRANSFORMS_REF || diff -u --color=always $ACTIVE_TRANSFORMS_REF $TESTS_ACTIVE_TRANSFORMS
- run: ccache -s
- run: ccache -sv
# delete old cache entry as we currently can't update it any other way
- env:
GH_TOKEN: ${{ github.token }}
if: steps.cache-restore.outputs.cache-hit && fromJSON(env.STORE_CACHE)
continue-on-error: true
run: gh cache delete -r ${{ github.ref }} ${{ steps.cache-restore.outputs.cache-primary-key }}
- if: fromJSON(env.STORE_CACHE)
uses: actions/cache/save@v5
with:
path: |
~/.cache/ccache
~/.ccache
key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- if: ${{ failure() }}
uses: actions/upload-artifact@v6
with:
@@ -180,22 +215,34 @@ jobs:
TEST: ${{ matrix.test }}
steps:
- uses: actions/checkout@v6
- uses: actions/cache@v5
- uses: actions/cache/restore@v5
id: cache-restore
with:
# path is different on newer systems
path: |
~/.cache/ccache
~/.ccache
key: ccache-${{ matrix.os }}-${{ env.CC }}-${{ matrix.test }}-${{ github.sha }}
key: ccache-${{ matrix.os }}-${{ env.CC }}-${{ matrix.test }}
restore-keys: |
ccache-${{ matrix.os }}-${{ env.CC }}-${{ matrix.test }}-
ccache-${{ matrix.os }}-${{ env.CC }}-
- run: |
sudo apt-get install -qq ccache
echo "PATH=/usr/lib/ccache:$PATH" >> $GITHUB_ENV
ccache -z
- uses: ./.github/actions/default
- run: ccache -s
- run: ccache -sv
# delete old cache entry as we currently can't update it any other way
- env:
GH_TOKEN: ${{ github.token }}
if: steps.cache-restore.outputs.cache-hit
continue-on-error: true
run: gh cache delete -r ${{ github.ref }} ${{ steps.cache-restore.outputs.cache-primary-key }}
- uses: actions/cache/save@v5
with:
path: |
~/.cache/ccache
~/.ccache
key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- if: ${{ failure() }}
uses: actions/upload-artifact@v6
with:
@@ -216,19 +263,28 @@ jobs:
- uses: actions/checkout@v6
# install tar and zstd before the cache action that requires them
- run: |
apk add ccache tar zstd
apk add ccache tar zstd github-cli
echo "PATH=/usr/lib/ccache/bin:$PATH" >> $GITHUB_ENV
- uses: actions/cache@v5
- uses: actions/cache/restore@v5
id: cache-restore
with:
path: ~/.cache/ccache
key: ccache-alpine-${{ github.sha }}
restore-keys: |
ccache-alpine-
key: ccache-alpine
- run: ccache -z
# don't use the default action as we don't want to build dependencies or install bash
- run: ./scripts/test.sh deps
- run: ./scripts/test.sh
- run: ccache -sv
# delete old cache entry as we currently can't update it any other way
- env:
GH_TOKEN: ${{ github.token }}
if: steps.cache-restore.outputs.cache-hit
continue-on-error: true
run: gh cache delete --repo ${{ github.repository }} -r ${{ github.ref }} ${{ steps.cache-restore.outputs.cache-primary-key }}
- uses: actions/cache/save@v5
with:
path: ~/.cache/ccache
key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- if: ${{ failure() }}
uses: actions/upload-artifact@v6
with: