Konstantin Shemyak - Tech Notes (Posts about x.509)

Minimal openssl.cnf

Konstantin Shemyak — Sat, 29 Apr 2017 17:00:00 GMT

Summary: Need to create X.509 certificate chain? This post explains how to do it with OpenSSL commands and gives minimal working examples of the configuration for typical test/demo cases.

A config file is needed

OpenSSL commands (that is, the ones invoked from the command line) are mostly controlled by their options. But in order to issue X.509 certificates or CSRs OpenSSL needs a configuration file. This file may contain no useful information, but is still required. This post gives and explains minimal configuration files for OpenSSL commands used for certificate generation: x509, req and ca.

OpenSSL installation should supply a default configuration file, often called openssl.cnf. Below, we'll use expressions openssl.cnf and "OpenSSL configuration file" interchangeably.

The topic of this post is the minimal configuration file. Sometimes the command will need more parameters than such minimum contains; in these cases they will be provided via command-line options. These options are kept at minimum too, so in real applications you'd likely need to pass more parameters - either via the config file or through the options.

Let's proceed to creating the certificate chain, i.e. one CA certificate and one "leaf" certificate issued by the CA.

Generate the private key

Generation of the private key can be included in some of the commands below, but for clarity let's create a private key with a separate command. To sound modern, pick an elliptic curve from openssl ecparam -list_curves with the name which sounds cute, such as secp521r1 (on a serious note - selection of the elliptic curve is out of scope of this post):

$ openssl ecparam -genkey -name secp521r1 -out privkey.pem

We'll use this same key for all certificates which appear later; of course such practice is acceptable only for demonstration or testing.

Create a self-signed CA certificate

This command needs a config file. Unless one is provided, OpenSSL will use the default, which may not contain what you want. There are two ways to provide a custom config:

-config option
OPENSSL_CONF environment variable

We'll use the first way to show the config file in a more explicit manner.

There are two OpenSSL subcommands which can be used to create a self-signed CA certificate: req -x509 and ca. Examples of both are below.

Using "req -x509" command

The minimum config file for this command is:

[ req ]
distinguished_name = req_dn

[ req_dn ]

Call this file openssl-min-req.cnf. With it, you can issue self-signed certificates:

$ openssl req -new -x509 -nodes -key privkey.pem -config openssl-min-req.cnf -subj "/CN=My self-signed CA certificate" -out ca.pem

If we do not specify the version explicitly or request any of X.509v3 extensions, then OpenSSL sets the version the certificate to 1.

Check with openssl x509 -purpose -in ca.pem and observe that various "CA" purposes come with Yes (WARNING code=3).

Surprisingly, for a V3 certificate, there seems to be no clear indication whether it is a CA certificate or not. Two extensions claim properties related to CA functionality. OpenSSL's C API function X509_check_ca returns non-zero when the certificate either has CA:TRUE in basicConstraints extension, or keyCertSign in keyUsage extension. Other software may follow other rules (for example, require CA:TRUE). To satisfy OpenSSL's interpretation, the minimal openssl.cnf can be like this:

[ req ]
distinguished_name = req_dn
x509_extensions = v3_ext

[ req_dn ]

[ v3_ext ]
basicConstraints = CA:true

Using "ca -selfsign" command

Below is the minimal openssl-ca.cnf which would do the job.

[ ca ]
default_ca      = CA_default

[ CA_default ]
database        = index.txt
serial          = serial.txt
policy          = policy_default

[ policy_default ]

For this command to succeed, three files referenced above must be created:

$ touch index.txt index.txt.attr
$ echo '01' > serial.txt

When done, self-signed CA certificate is created with this command:

$ openssl ca -config openssl-ca.cnf -selfsign -in csr.pem -keyfile privkey.pem -md default -out ca.pem -outdir . -days 365 -batch

Now we have the CA certificate. Next, create leaf certificates signed by it.

Create a certificate signed by your new CA

Create a CSR

A CSR (Certificate Signing Request) is made with req command. For it, the "minimum request openssl.cnf" is sufficient:

$ openssl req -new -config openssl-min-req.cnf -key privkey.pem -nodes -subj "/CN=Non-CA example certificate" -out csr.pem

Inspect the CSR with openssl req -text -noout -in csr.pem.

Having a CSR, the corresponding certificate can be issued using either x509 or ca commands. Below are examples for both.

Sign the CSR using "x509 -req" command

$ openssl x509 -req -in csr.pem -CA ca.pem -CAkey privkey.pem -CAcreateserial -out cert.pem

The x509 command seems to have no option -config, but it honors the environment variable OPENSSL_CONF. If the file is not readable, OpenSSL prints

WARNING: can't open config file: /usr/lib/ssl/openssl.cnf

and continues. For the example above, config file is not needed; and if it is not needed, you should not use it, because it may bring in items which you do not want. It looks like the way to "disable" config file processing is to set OPENSSL_CONF=some_nonexisting_file.

Sign the CSR using "ca" command

Previously mentioned openssl-ca.cnf works also for this case, and cannot be reduced.

$ openssl ca -config openssl-ca.cnf -cert ca.pem -keyfile privkey.pem -in csr.pem -out cert.pem -outdir . -md default -days 365 -batch

By default the ca command does not copy the X.509v3 extensions from the CSR (the ones specified in x509_extensions section of the [ req ] part in our example openssl-min-req.cnf) to the signed certificate. To do so, add copy_extensions = copy line to the CA section ([ CA_default ] in our example openssl-ca.cnf). Alternatively, the CA may add own extensions when signing a CSR - for example, set CA:FALSE. These should be listed in the section with the name given by the variable x509_extensions in the [ CA_default ] section.

Conclusion

To create certificate chains with OpenSSL, a configuration file is needed.
OpenSSL will use the default config file unless you provide another one via command-line option or an environment variable.
- Except that x509 -req is missing the option.
While the default may work for some cases, if you need any control over your certificates, you'll need to create the config file.
You get more control over the content of your certificates when starting with the bare minimum than with the default. The minimal examples are provided in this post.

OpenSSL PKCS7 verification and certificate "Extended Key Usage" extension

Konstantin Shemyak — Sat, 31 Dec 2016 20:34:22 GMT

Problem

You verify a signature of PKCS#7 structure with OpenSSL and get error

  unsupported certificate purpose

This post explains the reason for this error and ways to proceed.

Background

By "verify a signature", one probably means that:

The signature itself (e.g. an RSA block) taken over the corresponding data (or its digest) validates against the signing certificate.
Two sets of certificates are available, which we'd call "trusted certificates" and "chaining certificates". A chain from the signing certificate up to at least one of the trusted certificates can be built with the chaining certificates.
All certificates in this chain have "acceptable" X.509v3 extensions.

The first requirement is clear.

The second one is clear when the sets are defined. OpenSSL API requires them to be passed as parameters for the verification.

The last requirement relies on X.509v3 extensions, which are a terrible mess.

It's hard to provide a non-messy solution for a messy specification. Section CERTIFICATE EXTENSIONS in the OpenSSL manual for x509 subcommand has this passage:

The actual checks done are rather complex and include various hacks and
workarounds to handle broken certificates and software.

It looks like PKCS7 verification fell victim of these "hacks and workarounds".

OpenSSL certificate verification and X.509v3 extensions

Before getting to the topic (verifying PKCS#7 structures), look at how OpenSSL verifies certificates. Both command-line openssl verify and C API X509_verify_cert() have a notion of purpose, explained in the section CERTIFICATE EXTENSIONS of man x509. This notion seems to be particular to OpenSSL.

If the purpose is not specified, then OpenSSL does not check the certificate extensions at all.
Otherwise, for each purpose, OpenSSL allows certain combinations of the extensions.

The correspondence between OpenSSL's purpose and X.509v3 extensions is nothing like one-to-one. For example, purpose S/MIME Signing (or in short variant smimesign) requires that:

"Common S/MIME Client Tests" pass (description of how they translate to X.509v3 extension takes a long paragraph in man x509).
Either KeyUsage extension is not present, or it is present and contains at least one of digigalSignature and nonRepudiation flags.

For another example, there seems to be no OpenSSL command-line option for verify to require presense of Extended Key Usage bits like codeSigning. For that, one must use C API to separately check every extension bit.

So far, this sounds about as logical as it could be to somehow handle The Terrible Mess of X.509v3 extensions. OpenSSL CLI seems to have made an attempt to compose some "frequently used combinations" of the extensions and call them with own term "purpose".

OpenSSL PKCS#7 verification and X.509v3 extensions

By reason unknown yet to the author, OpenSSL uses a different strategy when verifying PKCS#7.

Command-line

There are two command-line utilities which can do that: openssl smime -verify and openssl cms -verify (S/MIME and CMS are both PKCS#7). Both accept -purpose option, which according to manual pages has the same meaning as for certificate verification. But it does not. These are the differences:

If no -purpose option is passed, both commands behave as though they received -purpose smimesign.
It is possible to disable this smimesign purpose checking by passing -purpose any.

C API

On the C API side, one is supposed to use PKCS7_verify() for PKCS#7 verification. This function also behaves as though it verifies with smimesign purpose. (see setting X509_PURPOSE_SMIME_SIGN in pk7_doit.c:919).

Similarly as with the command-line, it is possible to disable checking the extensions, although with more typing.

In the C API, the verification "purpose" is a property of X509_STORE, passed to PKCS7_verify(), which plays the role of the trusted certificate set.

Side note: manipulation of the parameters directly on the store was added only to OpenSSL 1.1.0 with X509_STORE_get0_param(X509_STORE *store). In earlier versions, an X509_STORE_CTX must have been created from the store and parameters manipulates with X509_STORE_CTX_get0_param(). BTW support for OpenSSL v1.0.1 has ended just on the day of this writing.

Possible reasoning

One might imagine reasoning like this: for openssl smime, smimesign is kind of "default purpose" and thus is implicitly required; and openssl cms is in fact an attempt to rewrite openssl smime, thus behaving in the same way.

Such behavior is fine for S/MIME, and is not what you would expect for anything else packed into PKCS#7.

Translating from OpenSSL's "purpose" to X.509v3 extensions, verification fails unless your signing certificate satisfies the two conditions:

If the Key Usage extension is present, then it must include the digitalSignature bit or the nonRepudiation bit.
If the Extended Key Usage extension is present, then it must include email protection OID.

In fact, the first condition is "reasonable": RFC5280 states in section "Key Usage" that

For example, when an RSA key should be used only to verify signatures on objects other than public key certificates and CRLs, the digitalSignature and/or nonRepudiation bits would be asserted.

PKCS#7 qualifies as "object other than public key certificates and CRLs". But the second condition is not relevant for anything else than S/MIME. (Of course, in the end it is your certificate practice policy which determines, what is accepted and what not; the above is just "common sense").

Demo

Prepare the files

Create a chain of certificates: self-signed "root", then an "intermediate" signed by the root, then a "signing" signed by the intermediate. Add some extendedKeyUsage extension to the signing, but do not add emailProtection. For example, add codeSigning.

Create appropriate OpenSSL config files, as explained in "minimum openssl.cnf" post.

Create requests for all the three:

  $ openssl req -config openssl-CA.cnf -new -x509 -nodes -outform pem -out root.pem -keyout root-key.pem
  $ openssl req -config openssl-CA.cnf -new -nodes -out intermediate.csr -keyout intermediate-key.pem
  $ openssl req -config openssl-signing.cnf -new -nodes -outform pem -out signing.csr -keyout signing-key.pem

Sign the intermediate and the signing certificates:

  $ mkdir -p demoCA/newcerts
  $ touch demoCA/index.txt
  $ echo '01' > demoCA/serial
  $ openssl ca -config openssl-CA.cnf -in intermediate.csr -out intermediate.pem -keyfile root-key.pem -cert root.pem
  $ openssl ca -config openssl-signing.cnf -in signing.csr -out signing.pem -keyfile intermediate-key.pem -cert intermediate.pem

Create some PKCS7 structure, signed with the signing certificate. The chain certificates must be provided during the verification, or embedded into the signature. Let's embed the intermediate certificate. (If there had been more than one certificate in the chain, they would need to be simply placed in one .pem file):

  $ echo 'Hello, world!' > data.txt
  $ openssl smime -sign -in data.txt -inkey signing-crlsign-key.pem -signer signing-crlsign.pem -certfile intermediate.pem -nodetach > signed-crlsign.pkcs7

We have everything ready for verifying.

Verification with command-line OpenSSL tools

Attempt to verify it:

  $ openssl smime -verify -CAfile root.pem -in signed-crlsign.pkcs7 -out /dev/null -signer signing-crlsign.pem 
  Verification failure
  139944505955992:error:21075075:PKCS7 routines:PKCS7_verify:certificate verify error:pk7_smime.c:336:Verify error:unsupported certificate purpose

Attempt to verify, skipping extension checks:

  $ openssl smime -verify -CAfile root.pem -in signed-crlsign.pkcs7 -out /dev/null -signer signing-crlsign.pem -purpose any
  Verification successful

Attempt to verify it, specifying the OpenSSL "purpose" which the signing certificate satisfies:

  $ openssl smime -verify -CAfile root.pem -in signed-crlsign.pkcs7 -out /dev/null -signer signing-crlsign.pem -purpose crlsign
  Verification successful

Verification with the C OpenSSL API

The code below is "demo", any real application would have at least to check return codes of all system calls and free any allocated resources. But it shows how the verification of PKCS#7 structure (unexpectedly) fails, and succeeds after setting the "purpose" which the signing certificate satisfies:

    #include <stdlib.h>
    #include <stdio.h>
    #include <fcntl.h>              /* open() */

    #include <openssl/bio.h>
    #include <openssl/err.h>
    #include <openssl/ssl.h>
    #include <openssl/pkcs7.h>
    #include <openssl/safestack.h>
    #include <openssl/x509.h>
    #include <openssl/x509v3.h>     /* X509_PURPOSE_ANY */
    #include <openssl/x509_vfy.h>

    int main(int argc, char* argv[]) {
      X509_STORE *trusted_store;
      X509_STORE_CTX *ctx;
      STACK_OF(X509) *cert_chain;
      X509 *root, *intermediate, *signing;
      BIO *in;
      int purpose, ret;
      X509_VERIFY_PARAM *verify_params;
      PKCS7 *p7;
      FILE *fp;
      int fd;

      SSL_library_init();
      SSL_load_error_strings();

      fd = open("signed-ext-no-smimesign.pkcs7", O_RDONLY);
      in = BIO_new_fd(fd, BIO_NOCLOSE);
      p7 = SMIME_read_PKCS7(in, NULL);

      cert_chain = sk_X509_new_null();

      fp = fopen("root.pem", "r");
      root = PEM_read_X509(fp, NULL, NULL, NULL);
      sk_X509_push(cert_chain, root);

      fp = fopen("intermediate.pem", "r");
      intermediate = PEM_read_X509(fp, NULL, NULL, NULL);
      sk_X509_push(cert_chain, intermediate);

      trusted_store = X509_STORE_new();
      X509_STORE_add_cert(trusted_store, root);

      fp = fopen("signing-ext-no-smimesign.pem", "r");
      signing = PEM_read_X509(fp, NULL, NULL, NULL);

      ret = PKCS7_verify(p7, cert_chain, trusted_store, NULL, NULL, 0);
      printf("Verification without specifying params: %s\n", ret ? "OK" : "failure");

      /* Now set a suitable OpenSSL's "purpose", or disable its checking.
       * Note: since OpenSSL 1.1.0, we'd not need `ctx`, but could just use:
       * verify_params = X509_STORE_get0_param(trusted_store); */

      ctx = X509_STORE_CTX_new();
      X509_STORE_CTX_init(ctx, trusted_store, signing, cert_chain);
      verify_params = X509_STORE_CTX_get0_param(ctx);
      purpose = X509_PURPOSE_get_by_sname("crlsign"); /* Or: purpose = X509_PURPOSE_ANY */
      X509_VERIFY_PARAM_set_purpose(verify_params, purpose);
      X509_STORE_set1_param(trusted_store, verify_params);

      ret = PKCS7_verify(p7, cert_chain, trusted_store, NULL, NULL, 0);
      printf("Verification with 'crlsign' purpose: %s\n", ret ? "OK" : "failure");
      return 0;
    }

If our policy requires crlSign Key Usage, then we can use this example code. What if the policy needs some extension combination for which there is no suitable OpenSSL "purpose" - for example, CodeSigning Extended Key Usage? In that case it would not be possible to do it with just one call to PKCS7_verify, but the extensions need to be checked separately.

Conclusion

If you use OpenSSL for verifying PKCS#7 signatures, you should check whether either the following holds:

Your signing certificate has Extended Key Usage extension, but no emailProtection bit.
Your signing certificate has KeyUsage extension, but no digitalSignature neither nonRepudiation OID.

If this is the case, then verification with OpenSSL fails even if your signature "should" verify correctly.

For checking signatures with command-line openssl smime -verify, a partial workaround can be adding option -purpose any. In this case OpenSSL will not check Extended Key Usage extensions at all. This can be acceptable or not by your verification policy.

-purpose option allows to check only for certain (although probably common) x509v3 extension combinations. OpenSSL defines a number of what it calls "purposes". If you need to check a combination which does not correspond to any of these "purposes", it must be done in a separate operation.

For checking signatures with C API PKCS7_verify(), the algorithm can be the following:

If your policy does not care about X.509v3 extensions, set your verification parameters to X509_PURPOSE_ANY.
Otherwise, check X509v3 extensions of the signing certificate as required by your policy (example). Set a custom verification callback with X509_STORE_CTX_set_verify_cb(), which might either ignore the "unsupported certificate purpose" error, i.e. X509_V_ERR_INVALID_PURPOSE, or have some more complicated logic.