IPA and Hardware Security Modules (HSMs)

Introduction

A hardware security module (HSM) is a device that provides physical and tamper-proof security for sensitive data. It is used to provision cryptographic keys. These keys cannot leave the device. They generally communicate using the PKCS#11 standard.

Because they use PKCS#11 it is possible to simulate them in software. One way is using the SoftHSM2 library. This does not provide the same level of protection as a hardware device but it is useful for development and prototyping.

An effort was made in 2019 with conjuction with some interested users to implement support for HSMs in IPA using SoftHSM2. The underlying CA, dogtagpki, supports several HSMs so much of the heavy lifting is already done. This effort brought IPA about 90% of the way towards supporting HSMs but some blocking bugs prevented full support. A user provided instructions on doing a single server installation, https://magnus-k-karlsson.blogspot.com/2019/08/installing-dogtag-on-fedora-30-with.html

This effort was reconstituted in the summer of 2022 to build on the work done in 2019.

NSS and tokens

Before we can dive into implementation we need to start with some terminology. Taken from the PKCS#11 spec:

Slot: A logical reader that potentially contains a token.

Token: The logical view of a cryptographic device defined by Cryptoki.

I like to think about these literally. Think of an ATM. It has a slot. The token is your card. On a real HSM a token is probably not removable. It’s harder to steal an entire device than a small card. But internally one or more slots is exposed containing one or more tokens.

Each slot and each token will have a unique name. The token name will be referenced all throughout this document.

NSS was built around the PKCS#11 specification so that all certificates and keys are stored in a slot. By default it is the internal NSS slot, the so-called soft token ( also called softokn). You don’t have to specify this slot when accessing keys and certificates on the soft token. It is implicit.

Any other PKCS#11 module will require you to specify the token it’s on and provide a PIN to access it.

Access to these slots and keys using PKCS#11 is provided by a shared library. So you need this library installed and the token to use initialized with a PIN before you can even start.

Installing

Installation is still a bit rough. These instructions are use unreleased code in my private repository. It is not production ready. There are only builds for Fedora 36.

The design of how it will be implemented in the IPA installers is under review. The below instructions use a workaround created by the 2019 team. This is not a long-term solution as it is too easy to mess up IMHO.

These instructions are going to revolve around using the SoftHSM2 library with software tokens. Using a hardware HSM instead should require just replacing the shared library and token name(s).

Start by enabling my custom repo with the necessary builds.

# dnf copr enable rcritten/freeipa

# dnf -y install freeipa-server-dns

Next we need to do some system setup so that the PKI subsystem can access the softhsm token. This is from Granting Permission to PKI System User:

# usermod pkiuser -a -G ods

Next we prepare our token. I’ve named it softhsm_token. You can name it whatever you’d like. The reason for runuser is so the token files are readable and owned by pkiuser.

# softhsm2-util --delete-token --token softhsm_token

# runuser -u pkiuser -- /usr/bin/softhsm2-util --init-token --free --pin password --so-pin password --label "softhsm_token"

Create the ini file to pass into the IPA installer.

[DEFAULT]

pki_hsm_enable=True

pki_hsm_libfile=/usr/lib64/pkcs11/libsofthsm2.so

pki_hsm_modulename=softhsm2

pki_token_name=softhsm_token

pki_token_password=password

pki_sslserver_token=internal

Install IPA with this HSM

# hostname ipa.example.test

# ipa-server-install -a password -p dmpassword -r EXAMPLE.TEST -U --setup-dns --allow-zone-overlap --no-forwarders -N --auto-reverse --pki-config-override=/root/pki.ini

Assuming everything went as planned you should now have an IPA installation that uses an HSM as its CA key storage.

Implementation Notes

The CA/pki-tomcat Server-Cert is stored in the database. This only serves requests so doesn’t need the security of the HSM. It also doesn’t work if you try to force it into the HSM because at some point it is exported as a PKCS#12 which won’t work because you can’t extract private key material from an HSM.

certutil output is going to look a bit strange.

# certutil -L -d /etc/pki/pki-tomcat/alias

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca CT,C,C
ocspSigningCert cert-pki-ca ,,
Server-Cert cert-pki-ca u,u,u
subsystemCert cert-pki-ca ,,
auditSigningCert cert-pki-ca ,,P

Where are the u’s you might ask? The u trust attribute indicates the presence of the private key. Since the key is on the HSM, it doesn’t show (except for the for-mentioned Server-Cert.

That these show at all is a side-effect of the way NSS handles trust flags. There is no PKCS#11 equivalent so they are stored in the NSS soft token instead.

Looking at the HSM token:

# certutil -L -d /etc/pki/pki-tomcat/alias -h softhsm_token

Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI

Enter Password or Pin for "softhsm_token":
softhsm_token:ocspSigningCert cert-pki-ca u,u,u
softhsm_token:caSigningCert cert-pki-ca CTu,Cu,Cu
softhsm_token:subsystemCert cert-pki-ca u,u,u
softhsm_token:auditSigningCert cert-pki-ca u,u,Pu

Here NSS melds the private key existence with the trust attributes from the soft token to give a more typical view of things.

You can’t mix tokens. Only one token per server is allowed. Technically PKI could use a different one for each key but I haven’t tested that and preventing regressions would likely be a nightmare. If you need it “d need a pretty strong use-case.

My personal repo

You might wonder what unreleased patches lurk in my unreleasted repo. Here is a general overview.

freeipa

IPA got some understanding of tokens in the 2019 effort but it was not complete. The certmonger tracking had to be updated to store the token. Similarly certmonger needs to know the user to run as so that when a certificate is issued the ownership is correct in softhsm.

When using an HSM we can’t use Custodia to retrieve keys for replica install.

I added some extended certificate handle so that freeipa-healthcheck can validate the installation.

certmonger

When using SoftHSM2 any files created need to be owned by the pkiuser otherwise the CA will not be able to read them. certmonger runs as root. As a workaround when saving certificate data it will do a setuid/setgid so that permissions are correct.

freeipa-healthcheck

healthcheck had no understanding of tokens at all. It gets the list of expected certificates from IPA. That exchange needed to be extended so that both the soft token and the HSM token were retrieved so that tracking and certificate trust attributes are as expected. This is not yet in my repo but I have a PR upstream.

NSS

When modifying certificate trust certutil needs to authenticate to the token holding the certificate. The problem is that the trust is stored in the soft token so that also needs to be authenticated. Have it fall back to the soft token when adding/modify trust.

pki

The installer code needed a fair bit of work due to permissions of the SoftHSM token files. A lot of calls need to be wrapped in runuser so that the resulting file permissions are ok. It is TBD whether this is problematic in a physical HSM installation. Most of these changes are already merged. If you want a detailed look just look at any recent merges by me.

Replica installation

I need to get a patch merged with the CA in order for replica installation to work. I’ll either update this post or create a new one when that is done.

KRA installation

Installing a KRA during intial install, --setup-kra, or afterward, ipa-kra-install, works for me.

Advertisement

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s