X.509 is the first thing that comes to mind when discussing digital certificates. After all, it is the most widely used digital certificate in the PKI ecosystem and is the core component of SSL/TLS protocols, the technology that powers HTTPS. X.509 was first released on 25 November 1988 and is powerful, extensible and widely supported. But it’s not the only certificate format available out there. For example, the popular email encryption program PGP uses a custom certificate format instead of X.509. And PGP is not the only one to avoid X.509 — the most widely used remote access server OpenSSH also uses a custom certificate format. They call it certkeys!
OpenSSH has supported certificate-based authentication since version 5.4, but despite X.509 being widely supported, OpenSSH did not base the certificates in X.509 format but instead invented a simple but capable and compatible enough format for certificate-based authentication. Why? It is questionable, but the reasons must have been well-founded for sure!
In this blog post, we will explore how OpenSSH’s certificate compares with X.509, including a few strengths and weaknesses of the OpenSSH implementation.
The implementation details of OpenSSH certificate-based authentication are well-described in the PROTOCOL.certkeys document, but I’ll highlight the important points below:
Let’s first try to understand how OpenSSH implements certificate-based authentication. In OpenSSH, the CA certificate is just public and private key pairs with additional identity and constraints data. The private key of the CA is used to sign user and host (SSH server) certificates. Once the keys are signed, they are distributed to users and hosts, respectively. The public key of the CA is copied to the SSH server, which is used to verify the user’s certificate.
For user authentication, the SSH client presents the user certificate (signed by CA) to the SSH server in each new SSH connection. The SSH server validates the certificate by checking it against the CA’s public key. Along with the signature validation, the server will also check if the user certificate is not expired and if it violates security constraints. Access is granted upon successful validation of the certificate.
This authentication flow is similar to public-key authentication with just additional verification of trust based on a signature signed by a trusted CA and additional constraints of expiry dates and identity attributes. Under the hood, this logical extension means that implementation for certificate-based authentication became as simple as extending the public-key authentication protocol as specified in RFC 4252. This is done via SSH protocol extension and adding new keys for certificates including following key types:
Keeping the certificate format minimal helps reduce the attack surface. X.509 is a behemoth when it comes to certificate attributes and extensibility. Supporting X.509 certificates means implementing full-fledged X.509 PKI. Given X.509 is designed to be universal, as such it carries loads of features that simply may not be required for SSH authentication. Reducing attack surfaces is always a welcomed approach for maintaining high security.
This is not explained in the design document shared above, but I’ve seen this mentioned many times in discussion forums. Historically, OpenSSH crypto libraries were all based on OpenSSL’s crypto library, and as such, any feature sets on the OpenSSH protocol should be first supported by OpenSSL. Implementing a custom format allows OpenSSH to maintain and support better solutions without the need to wait for implementations upstream.
Custom implementations allow OpenSSH to support best practices for CA and certificate management. Maintaining separate CA for a user and host authentication is an expected workflow in OpenSSH. Algorithms such as AES, ChaCha20, RSA, ECDSA, and Ed25519 are readily available.
Simplicity comes with a price! OpenSSH certificates lack the broader customizability that X.509 supports.
X.509 are widely supported, so chances are that your organization already maintains trust with an internal or external X.509 CA. But to support certificate-based authentication, you will need to maintain another CA that can handle SSH certificate signing and issuance.
OpenSSH has no mechanism to support cross-signing CA and managing multiple CA hierarchies besides the notion of user and host CA certificates. This can pose a challenge in a large-scale implementation where multiple intermediate CA can come in handy during CA rotation or CA migration.
Below is a high-level overview of X.509 and OpenSSH certificate properties.
|X.509 CERTIFICATE||OPENSSH CERTIFICATE|
|Use Case||Mostly universal. SSL/TLS, S/MIME, EAP-TLS etc. Can be used with SSH protocol.||Only for SSH|
|Key generation algorithms||DDSA, RSA, EcDSA, EdDSA (limited support)||DSA, RSA, ECDSA, EDDSA|
|Signature algorithms||MD2, MD5, SHA-1, SHA-256, SHA-384, SHA-512. MD2, MD5 and SHA-1 are considered insecure and should be avoided.||SHA-256, SHA-512|
|Certificate Extensibility (attaching additional metadata)||Using certificate extension||Using certificate extension|
|Certificate encoding format||PEM, DER||Custom public key format. Can be exported as PEM.|
|Popular Tools to generate certificate||OpenSSL client tool and libraries, X.509 compatible libraries.||
Let’s look at a sample OpenSSH certificate created using OpenSSH
Let’s look at a sample X.509 certificate created using OpenSSL below:
In this post, we explained why OpenSSH implements its own certificate format and how it differs from X.509 standard. The OpenSSH implementation is simple enough to cater to the requirements of certificate-based authentication as implemented by OpenSSH. But maintaining two different types of CA can be challenging for organizations. Should OpenSSH ultimately support X.509? I like the fact that it’s simple yet secure enough with a reduced attack surface.
Remoteler implements both X.509 and OpenSSH CA management which makes it easy to manage certificate-based authentication in both small and large-scale infrastructure deployments.