If it takes a while to start one of the threads, another thread might already
have passed the usleep() call previously used and re-enabled cancelability
so that the loop that checked for it would never terminate.
With the previous approach we'd require at least an additional pointer
per item to store them in a list (15-18% increase in the overhead per
item). Instead we switch from handling collisions with overflow lists to
an open addressing scheme and store the actual table as variable-sized
indices pointing into an array of all inserted items in their original
order.
This can reduce the memory overhead even compared to the previous
implementation (especially for smaller tables), but because the array for
items is preallocated whenever the table is resized, it can be worse for
certain numbers of items. However, avoiding all the allocations required
by the previous design is actually a big advantage.
Depending on the usage pattern, the performance can improve quite a bit (in
particular when inserting many items). The raw lookup performance is a bit
slower as probing lengths increase with open addressing, but there are some
caching benefits due to the compact storage. So for general usage the
performance should be better. For instance, one test I did was counting the
occurrences of words in a list of 1'000'000 randomly selected words from a
dictionary of ~58'000 words (i.e. using a counter stored under each word as
key). The new implementation was ~8% faster on average while requiring
10% less memory.
Since we can't remove items from the array (would change the indices of all
items that follow it) we just mark them as removed and remove them once the
hash table is resized/rehashed (the cells in the hash table for these may
be reused). Due to this the latter may also happen if the number of stored
items does not increase e.g. after a series of remove/put operations (each
insertion requires storage in the array, no matter if items were removed).
So if the capacity is exhausted, the table is resized/rehashed (after lots
of removals the size may even be reduced) and all items marked as removed
are simply skipped.
Compared to the previous implementation the load factor/capacity is
lowered to reduce chances of collisions and to avoid primary clustering to
some degree. However, the latter in particular, but the open addressing
scheme in general, make this implementation completely unsuited for the
get_match() functionality (purposefully hashing to the same value and,
therefore, increasing the probing length and clustering). And keeping the
keys optionally sorted would complicate the code significantly. So we just
keep the existing hashlist_t implementation without adding code to maintain
the overall insertion order (we could add that feature optionally later, but
with the mentioned overhead for one or two pointers).
The maximum size is currently not changed. With the new implementation
this translates to a hard limit for the maximum number of items that can be
held in the table (=CAPACITY(MAX_SIZE)). Since this equals 715'827'882
items with the current settings, this shouldn't be a problem in practice,
the table alone would require 20 GiB in memory for that many items. The
hashlist_t implementation doesn't have that limitation due to the overflow
lists (it can store beyond it's capacity) but it itself would require over
29 GiB of memory to hold that many items.
The main intention here is that we can change the hashtable_t
implementation without being impeded by the special requirements imposed
by get_match() and sorting the keys/items in buckets.
This can improve negative lookups, but is mostly intended to be used
with get_match() so keys/items can be matched/enumerated in a specific
order. It's like storing sorted linked lists under a shared key but
with less memory overhead.
There are a ton of libsoup/GLib-related "leaks" that we can't whitelist
and with leak detective active there is a delay that interestingly doesn't
happen with soup_session_sync_new(), so tests failed with a timeout (actually
they hung due to the lock in the fetcher manager).
On Travis, the curl plugin is used for the tests, so that's not an issue
there (and without LD the tests complete quickly and successfully).
If strings are missing (e.g. because the last value of a range changed
unknowingly or adding a string was simply forgotten) compilation will
now fail.
This could be problematic if the upper limit is out of our control (e.g.
from a system header like pfkeyv2.h), in which case patches might be
required on certain platforms (enforcing at least, and not exactly, the
required number of strings might also be an option to compile against
older versions of such a header - for internal enums it's obviously
better to enforce an exact match, though).
This avoids having to call strip_dh() in child_cfg_t::get_proposals().
It also inverts the ALLOW_PRIVATE flag (i.e. makes it SKIP_PRIVATE) so
nothing has to be supplied to clone complete proposals.
During proposal selection with ike/child_cfgs a couple of boolean
variables can be set (e.g. private, prefer_self, strip_dh). To simplify
the addition of new parameters, these functions now use a set of flags
instead of indiviual boolean values.
Signed-off-by: Thomas Egerer <[email protected]>
Our own implementation ignores NULL values, however, explicit_bzero()
can't handle that, as indicated by the `__nonnull ((1))` attribute in the
function's signature in string.h, and causes a segmentation fault. This
was noticed in one of the unit tests for NewHope. Since we usually use
memwipe() via chunk_clear(), which already ignores NULL pointers, this
is not that much of an issue in practice.
Fixes: 149d1bbb05 ("memory: Use explicit_bzero() as memwipe() if available")
The behavior is undefined if this happens (RFC 7296, section 2.13).
Instead of switching to the non-counter mode, or letting the counter
wrap, this makes it clear that the usage was not as intended.
Same issue with signature malleability as with Ed25519 and apparently
OpenSSL doesn't even explicitly verify that the most significant 10 bits
are all zero.
As per RFC 8032, section 5.1.7 (and section 8.4) we have to make sure s, which
is the scalar in the second half of the signature value, is smaller than L.
Without that check, L can be added to most signatures at least once to create
another valid signature for the same public key and message.
This could be problematic if, for instance, a blacklist is based on hashes
of certificates. A new certificate could be created with a different
signature (without knowing the signature key) by simply adding L to s.
Currently, both OpenSSL 1.1.1 and Botan 2.8.0 are vulnerable to this, which is
why the unit test currently only warns about it.
Botan only allows RSA generating keys >= 1,024 bits, which makes
the RSA test suite fail. It is questionable whether it makes
sense to test 768 bit RSA keys anymore. They are too weak
from today's perspective anyway.
This requires config changes if filelog is used with a path that
contains dots. This path must now be defined in the `path` setting of an
arbitrarily named subsection of `filelog`. Without that change the
whole strongswan.conf file will fail to load, which some users might
not notice immediately.
For inbound processing, it can be rather useful to apply the mark to the
packet in the SA, so the associated policy with that mark implicitly matches.
When using %unique as match mark, we don't know the mark beforehand, so
we most likely want to set the mark we match against.
%unique (and the upcoming %same key) are usable in specific contexts only.
To restrict the user from using it in other places where it does not get the
expected results, reject such keywords unless explicitly allowed.
There was a potential chance for a race condition if the ensured section
was purged for some reason before using it later.
This also changes the behavior for NULL/empty strings via load_string*
with merge == FALSE, which now purges the config/section.
Similar to the `also` keyword in ipsec.conf, the new syntax allows adding
one or more references to other sections, which means all the settings and
subsections defined there are inherited (values may be overridden, even
with an empty value to clear it).
It's important to note that all subsections are inherited, so if this is
used to reference a connection in swanctl.conf all auth rounds and
children are inherited. There is currently no syntax to limit the
inclusion level or clear inherited sections (but as mentioned, settings
in those inherited sections may be overridden).
Another property is that inherited settings or sections always follow
explicitly defined entries in the current section when they are enumerated.
This is relevant if the order is important (e.g. for auth rounds if `round`
is not specified).
References are evaluated dynamically at runtime, so referring to
sections later in the config file or included via other files is no
problem.
The colon used as separator to reference other sections may be used in
section names by writing :: (e.g. for Windows log file paths).
This is based on a patch originally written in 2016.
This was quite confusing previously: While calling insert_before()
and then remove_at() properly replaced the current item, calling them the
other way around inserted the new item before the previous item because
remove_at() changed the enumerator's position to the previous item.
The behavior in corner cases (calling the methods before or after
enumeration) is also changed slightly.
Tobias Brunner