OCSP Stapling on Nginx and Apache webserver

2020-06-01

Web Servers Networking Security

Have you error 'ssl_stapling_init_cert' like this in your apache log file?

Can't retrieve issuer certificate!

Or error:

OCSP_basic_verify() failed (SSL: error:27069065:OCSP routines:OCSP_basic_verify:certificate verify error:Verify error:unable to get local issuer certificate) while requesting certificate status

in your nginx error file? Probably you have a misconfigured web server for OCSP STAPLING support"

What is OCSP ?

To understand OCSP stapling, it is necessary to understand OCSP, the Online Certificate Status Protocol. OCSP is a protocol for determining whether a certificate is revoked (for instance, because its private key was compromised). Every time a browser connects to an HTTPS website, it contacts the OCSP responder specified in the SSL certificate, and asks if the certificate is revoked. If the responder replies that the certificate is revoked, the browser blocks the page from loading.

OCSP was created as an alternative to CRL to reduce the SSL negotiation time. With CRL (Certificate Revocation List) the browser downloads a list of revoked certificate serial numbers and verifies the current certificate, which increases the SSL negotiation time.

In OCSP the browser sends a request to a OCSP URL and receives a response containing the validity status of the certificate. The following screenshot shows the OCSP URI of digitalocean.com.

OCSP URL

What is OCSP Stapling ?

OCSP has major two issues: privacy and heavy load on CA's servers.

Since OCSP requires the browser to contact the CA to confirm certificate validity it compromises privacy. The CA knows what website is being accessed and who accessed it.
If a HTTPS website gets lots of visitors the CA's OCSP server has to handle all the OCSP requests made by the visitors.

OCSP stapling fixes these two problems by having the web server make the OCSP request and including ("stapling") the response along with the certificate in the SSL handshake. The browser can use the response from the server instead of making its own OCSP request, and since the server can cache the OCSP response and reuse it with future connections, it doesn't slow down page load times.

Here is how OCSP stapling works:

A web server hosting an SSL-encrypted website queries the certificate vendor. The vendor responds with the status of the certificate and a digitally signed time-stamp. Digitally signing the response makes it difficult for the web server to modify it.
When a web browser connects to the server, the server bundles (or "staples") the vendor’s signed time-stamp with the SSL certificate.
The browser verifies the time-stamp. Since the time-stamp is signed by the vendor, the browser can trust the time-stamp to provide a valid status.
Based on the OCSP response, the browser either opens the page or shows an error message to the user.

OCSP STAPLING

Check for OCSP stapling support

OCSP stapling is supported on

Apache HTTP Server (>=2.3.3)
Nginx (>=1.3.7)

Apache:

~] apache2 -v
Server version: Apache/2.4.57 (Debian)
Server built:   2023-04-13T03:26:51

Nginx:

~] nginx -v
nginx version: nginx/1.18.0

CentOS/Fedora users replace apache2 with httpd.

Retrieve the CA bundle

Retrieve the root CA and intermediate CA's certificate in PEM format and save them in a single file. This is for StartSSL's Root and Intermediate CA certificates.

~] cd /etc/ssl
/etc/ssl ~] wget -O - https://www.startssl.com/certs/ca.pem https://www.startssl.com/certs/sub.class1.server.ca.pem | tee -a ca-certs.pem> /dev/null

If your CA provides certificates in DER format convert them to PEM. For example DigiCert provides certificates in DER format. To download them and convert to PEM use the following commands:

~] cd /etc/ssl
wget -O - https://www.digicert.com/CACerts/DigiCertHighAssuranceEVRootCA.crt | openssl x509 -inform DER -outform PEM | tee -a ca-certs.pem > /dev/null
wget -O - https://www.digicert.com/CACerts/DigiCertHighAssuranceEVCA-1.crt | openssl x509 -inform DER -outform PEM | tee -a ca-certs.pem > /dev/null

Both sets of commands use tee to write to the file, so you can use sudo tee if logged in as a non-root user.

CA-Bundle in linux distribution

When your server certificates files contains all intermediate certificates you need, so you can use also a CA-bundle from your linux distribution. It is almost in file /etc/ssl/certs/ca-certificates.crt

Configuring OCSP stapling on Nginx

Edit the SSL virtual hosts file and place the following directives inside the server {} section.

/etc/nginx/sites-enabled/example.com.ssl

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/ssl/private/ca-certs.pem;

the complete virtual host file will look like this:

server {

        listen   443;
        server_name example.org;

        root /usr/share/nginx/www;
        index index.html index.htm;

        ssl on;
        ssl_certificate /etc/nginx/ssl/example.org/server.crt;
        ssl_certificate_key /etc/nginx/ssl/example.org/server.key;

        ssl_stapling on;
        ssl_stapling_verify on;
        ssl_trusted_certificate /etc/ssl/private/ca-certs.pem;
}

Do a configtest to see if everything is correct:

service nginx configtest
# or in debian linux distros:
/etc/init.d/nginx configtest

Then reload the nginx service.

service nginx reload
# or in debian linux distros:
/etc/init.d/nginx reload

Access the website and check the error log.

tail /var/log/nginx/error.log

If the file defined in ssl_trusted_certificate is missing a certificate an error similar to the following is displayed:

2019/07/17 17:38:16 [error] 15810#15810: OCSP_basic_verify() failed (SSL: error:27069065:OCSP routines:OCSP_basic_verify:certificate verify error:Verify error:unable to get local issuer certificate) while requesting certificate status, responder: ocsp.startssl.com, peer: 93.184.220.29:80, certificate: "/etc/ssl/wildcard.sherlogtrace.cz.pem"
2019/07/17 17:39:20 [error] 84601#84601: OCSP_basic_verify() failed (SSL: error:27069065:OCSP routines:OCSP_basic_verify:certificate verify error:Verify error:self signed certificate in certificate chain) while requesting certificate status, responder: ss.symcd.com, peer: 23.63.139.27:80, certificate: "/etc/ssl/localiza-telefonica.com.pe.pem"
2019/07/17 17:39:45 [error] 10068#10068: OCSP_basic_verify() failed (SSL: error:27069065:OCSP routines:OCSP_basic_verify:certificate verify error:Verify error:unable to get local issuer certificate) while requesting certificate status, responder: ocsp2.globalsign.com, peer: 104.18.20.226:80, certificate: "/etc/ssl/gps.movistar.com.ar.pem"

Configuring OCSP Stapling on Apache

Edit the SSL virtual hosts file and place these lines inside the <VirtualHost></VirtualHost> directive.

/etc/apache2/sites-enabled/example.com-ssl.conf

SSLCACertificateFile /etc/ssl/ca-certs.pem
SSLUseStapling on

A cache location has to be specified outside .

/etc/apache2/mods-enabled/ssl.conf

SSLStaplingCache shmcb:/tmp/stapling_cache(128000)

Apache, the virtual host file will look this:

/etc/apache2/sites-enabled/example.com-ssl.conf

<IfModule mod_ssl.c>
    SSLStaplingCache shmcb:/tmp/stapling_cache(128000)
    <VirtualHost *:443>

            ServerAdmin webmaster@localhost
            ServerName example.com
            DocumentRoot /var/www

            SSLEngine on

            SSLCertificateFile /etc/apache2/ssl/example.com/apache.crt
            SSLCertificateKeyFile /etc/apache2/ssl/example.com/apache.key

            SSLCACertificateFile /etc/ssl/ca-certs.pem
            SSLUseStapling on
    </VirtualHost>
</IfModule>

Do a configtest to check for errors.

apachectl -t

Reload if Syntax OK is displayed:

service apache2 reload
# or in debian linux distros:
/etc/init.d/apache2 reload

Access the website and check the error log.

tail /var/log/apache2/error.log

If the file defined in the SSLCACertificateFile directive is missing, a certificate an error similar to the following is displayed.

[Fri Jul 12 23:36:44.055900 2019] [ssl:error] [pid 1491:tid 139921007208320] AH02217: ssl_stapling_init_cert: Can't retrieve issuer certificate!
[Fri Jul 12 23:36:44.056018 2019] [ssl:error] [pid 1491:tid 139921007208320] AH02235: Unable to configure server certificate for stapling

Testing OCSP Stapling

Two methods will be explained to test if OCSP stapling is working - the openssl command-line tool and SSL test at Qualys .

The OpenSSL command

This command's output displays a section which says if your web server responded with OCSP data. We grep this particular section and display it.

~] echo QUIT | openssl s_client -connect www.mybluelinux.com:443 -status 2> /dev/null | grep -A 17 'OCSP response:' | grep -B 17 'Next Update'

Replace www.mybluelinux.com with your domain name. If OCSP stapling is working properly the following output is displayed.

OCSP response: 
======================================
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 0CDB6C82490F4A670AB814EE7AC4485288EB5638
    Produced At: Jul 17 15:50:48 2019 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 498942A04F69BCF1F6789F17352EEE8CFE3BE46A
      Issuer Key Hash: 0CDB6C82490F4A670AB814EE7AC4485288EB5638
      Serial Number: 07568396CD2916D4577C54B7688058E0
    Cert Status: good
    This Update: Jul 17 15:50:48 2019 GMT
    Next Update: Jul 24 15:05:48 2019 GMT

No output is displayed if OCSP stapling is not working.