openssl: Handle BoringSSL-style ASN1_INTEGERs in cert serials

OpenSSL stores the serial number for an X509 certificate as an
`ASN1_INTEGER` type. Within BoringSSL (and AWS-LC), the library
represents the value of zero as an empty array [1] which is different
from OpenSSL which represents it as the 1-byte array [0x00]. Though the
value of zero for the certificate serial number is illegal under
X.509 [2], we need to handle/encode it consistently within strongSwan.
From 18082ce2b0 ("certificates: Retrieve serial numbers in canonical
form"), we infer that the canonical representation of the zero serial
is [0x00]. To do this, we introduce `openssl_asn1_int2chunk` to
complement the existing string version that allows us to handle the
special case for zero instead of always returning a reference to the
library-dependent encodings.

References strongswan/strongswan#1907
Closes strongswan/strongswan#2138

[1] https://github.com/google/boringssl/commit/bdc35b63617f78037768f4897d8835696f02181a
[2] https://datatracker.ietf.org/doc/html/rfc5280#section-4.1.2.2
This commit is contained in:
Gerardo Ravago
2024-03-05 08:51:16 +01:00
committed by Tobias Brunner
parent 06afb5f109
commit 44e241fccc
4 changed files with 33 additions and 2 deletions
@@ -125,6 +125,14 @@ chunk_t openssl_asn1_obj2chunk(const ASN1_OBJECT *asn1);
*/
chunk_t openssl_asn1_str2chunk(const ASN1_STRING *asn1);
/**
* Convert an OpenSSL ASN1_INTEGER to a chunk.
*
* @param asn1 asn1 integer to convert
* @return chunk, pointing into asn1 integer
*/
chunk_t openssl_asn1_int2chunk(const ASN1_INTEGER *asn1);
/**
* Convert an openssl X509_NAME to a identification_t of type ID_DER_ASN1_DN.
*