7.11.3 Example – Hybrid Public Key Encryption

Hybrid Public Key Encryption (HPKE) is a hybrid encryption scheme defined in RFC 9180 [15]. It provides public-key encryption of plaintexts for the recipient public key and supports entity authentication.

HPKE is composed of a public-key cryptosystem, a KDF used for key derivation, and an encryption algorithm that offers Authenticated Encryption with Associated Data (AEAD). We will cover AEAD in detail in Chapter 15, Authenticated Encryption.

Classical hybrid encryption, illustrated in Figure 7.7, uses public-key cryptography to transport or agree on a symmetric key. In contrast, if Bob wants to send Alice an encrypted message using HPKE, he generates a symmetric key k, encapsulates it with Alice’s public key, and sends the encapsulated k alongside one or more ciphertexts of arbitrary size encrypted under k.

In its base mode, HPKE encrypts the payload data Bob wants to send to Alice to a public key [100] as shown in Figure 7.8. The base mode is the most commonly used HPKE mode.

Bob generates a shared secret key k and encrypts the plaintext message m that he wants to send Alice under key k. Bob then encrypts the key k using Alice’s public key PubKA and sends this encapsulated key alongside the resulting ciphertext c to Alice.

Upon receiving the encrypted data c and the encapsulated shared secret key, Alice uses her private key PrivKA to recover the symmetric key k. To do this, she decrypts the encapsulated key k to k = dPrivKA(k). She then uses k to decrypt the plaintext message m.

Figure 7.8: Base mode of HPKE hybrid encryption scheme

We conclude this section with a general discussion of the use of hybrid cryptosystems today.

7.11.4 Hybrid cryptosystems in modern cryptography

Most popular cryptographic systems in use today are hybrid cryptosystems. TLS is a prime example of a hybrid cryptosystem – it uses public-key cryptography (more specifically, Diffie-Hellman on Elliptic Curves (ECDH)) to agree on a shared secret key required to establish a secure communication channel between Alice and Bob and then, as we will learn in Part 2, Off the Record, employs symmetric key cryptography to encrypt the bulk data.

Another example is the well-known Secure Shell (SSH) protocol, which was introduced in 1996 [203]. SSH uses public-key cryptography (for instance, the Diffie-Hellman key agreement protocol) for the symmetric key exchange and a DEM based on symmetric-key cryptography for bulk data encryption. Just like TLS, SSH acts on top of the transport layer.

The OpenPGP and PKCS #7 file formats, as well as protocols such as SMIME, are further examples of hybrid cryptosystems. As yet another example, a hybrid cryptosystem called ECIES is used to store the secret keys in the secure enclave of iOS devices [10].

7.12 Summary

In this chapter, we introduced the mathematical foundations of public-key cryptosystems and looked in detail at the two most important examples, the Diffie-Hellman key exchange protocol and the RSA cryptosystem. We also investigated how exactly public-key cryptography is used within TLS.

By now, you should be aware of a very substantial difference between Diffie-Hellman and RSA: while RSA has to work with integers, the Diffie-Hellman protocol works in principle with any abelian group 𝔾. The difficulty of the discrete logarithm problem, which lies at the core of the Diffie-Hellman protocol, varies from group to group. If we can find a group where it is especially difficult, the corresponding key lengths could be shorter in that group. This fact is what makes elliptic curves so attractive in modern cryptography. They are the topic of our next chapter.