Clean up leak-detective whitelist for newer Botan versions but add
`botan_private_key_load` as `botan_privkey_load*` won't show up anymore
without bfd-backtraces due to inlining if we don't call it directly.
Ref: https://github.com/randombit/botan/discussions/4868
AWS-LC (and likely BoringSSL) uses thread specific data to store internal
library state which gets freed via a registered destructor when the thread
terminates. If this thread happens to be the main thread, which runs the
leak-detective evaluation, the detective won't observe the corresponding free
of the related memory and erroneously reports it as a leak.
The two places this happens are:
- `RAND_bytes` for storing internal RNG state.
- `ERR_put_error` for storing the per-thread OpenSSL error queue.
References strongswan/strongswan#1907Closesstrongswan/strongswan#2147
systemd seems to use this and if we indirectly use libraries provided
by it, which can e.g. happen via getgrnam_r() and nss-systemd, this may
be called on pointers returned by leak detective's malloc(), which will
not point to the original start of the block and cause a segmentation
fault.
Closesstrongswan/strongswan#2045
The all-zero Ed25519 public key is rejected by botan_pubkey_check_key()
when the key is loaded.
Note that Botan 3 requires GCC 11 or CLANG 14, i.e. can't easily be built
on Debian bullseye or Ubuntu 20.04.
The thread-local storage function gets flagged via various botan FFI
functions when using Botan 3, whitelist that instead of all of them.
There are a ton of libsoup/GLib-related "leaks" that we can't whitelist
and with leak detective active there is a delay that interestingly doesn't
happen with soup_session_sync_new(), so tests failed with a timeout (actually
they hung due to the lock in the fetcher manager).
On Travis, the curl plugin is used for the tests, so that's not an issue
there (and without LD the tests complete quickly and successfully).
Checking for whitelisted functions in every backtrace is not very
efficient. And because OpenSSL 1.1 does no proper cleanup anymore until
the process is terminated there are now a lot more "leaks" to ignore.
For instance, in the openssl-ikev2/rw-cert scenario, just starting and
stopping the daemon (test vectors are checked) now causes 3594 whitelisted
leaks compared to the 849 before. This prolonged the shutdown of the
daemon on each guest in every scenario, amounting to multiple seconds of
additional runtime for every affected scenario. But even with this
patch there is still some overhead, compared to running the scenarios on
jessie.
This also changes how unknown/corrupted memory is handled in the free()
and realloc() hooks in general.
Incorporates changes provided by Thomas Egerer who ran into a similar
issue.
Due to the mangled C++ function names it's tricky to be more specific. The
"leaked" allocations are from a static hashtable containing EC groups.
There is another leak caused by the locking allocator singleton
(triggered by the first function that uses it, usually initialization of
a cipher, but could be a hasher in other test runners), but we can avoid
that with a Botan config option.
In case an external thread calls into our code and logs messages, a thread
object is allocated that will never be released. Even if we try to clean
up the object via thread value destructor there is no guarantee that the
thread actually terminates before we check for leaks, which seems to be the
case for the Ada Tasking threads.
Some of these are pretty broad, so maybe an alternative option is to
not use the soup plugin in the openssl-ikev2/rw-suite-b* scenarios. But
the plugin is not tested anywhere else so lets go with this for now.
If a function we whitelist allocates memory while leak detective is enabled
but only frees it after LD has already been disabled, free() will get called
with invalid pointers (not pointing to the actually allocated memory by LD),
which will cause checks in the C library to fail and the program to crash.
This tries to detect such cases and calling free with the correct pointer.
Lots of static data is allocated in this function, which isn't freed until
the library is unloaded (we can't call OPENSSL_cleanup() as initialization
would fail when calling it again later). When enabling the leak
detective the test runner eventually crashes as all the data allocated during
initialization has an invalid size when freed after leak detective has been
unloaded.
Newer versions of GCC are too "smart" and replace a call to malloc(X)
followed by a call to memset(0,X) with a call co calloc(), which obviously
results in an infinite loop when it does that in our own calloc()
implementation. Using `volatile` for the variable storing the total size
prevents the optimization and we actually call malloc().
This prevented `stroke memusage` from reporting the leaks on the
console. Instead, they were sent to the callbacks set up by libstrongswan.
Fixes a426851f63 ("leak-detective: Use callback functions to report
leaks and usage information").
gcry_check_version() does not free statically allocated resources. However,
we can't whitelist it in some versions, as it is not a resolvable symbol name.
Instead, whitelist our own plugin constructor function.
If realloc return a pointer value different from the value to be
reallocated, a double free can occur in this context.
Signed-off-by: Thomas Egerer <[email protected]>
Tobias Brunner