Skip to content

Chapter 5: KEMs vs key agreement vs public-key encryption

KEMs, key agreement, and public-key encryption solve three different problems. A key-agreement protocol such as Diffie-Hellman lets two parties who share no secret derive one from messages an eavesdropper can read, and both parties contribute randomness to it. A public-key encryption (PKE) scheme lets a sender deliver a specific message, for example a key the sender already chose, that only the holder of the private key can read. A key-encapsulation mechanism (KEM) has a different interface. Its encapsulation algorithm takes the receiver’s public key and outputs a shared key together with a ciphertext. Unlike encryption, it takes no message from the caller. The receiver recovers the same key from the ciphertext with its private key. Post-quantum key establishment is standardized in this third form: NIST published ML-KEM directly as a KEM (National Institute of Standards and Technology, 2025b).

The chapter’s second question is what makes the last two secure, and correct decryption is not the answer. Textbook RSA decrypts every ciphertext correctly, yet an attacker can test guesses at its plaintext using only the public key. A randomized scheme built from this chapter’s own Diffie-Hellman resists that test and still gives its plaintext away to a different attacker. What separates the two attackers is what they can do. An eavesdropper only reads ciphertexts. An active attacker can also send ciphertexts of its own to the receiver and learn something from the result, and one such query breaks the Diffie-Hellman-based scheme, which no known classical eavesdropping attack breaks at real sizes.

The chapter writes both attackers down as security definitions and then builds the standard fix for the second, the Fujisaki-Okamoto transform. It takes an encryption scheme that resists eavesdroppers and produces a KEM that also resists the active attacker, by letting the receiver detect any ciphertext that is not what an honest sender would have produced for its own plaintext. ML-KEM is built this way, and Chapter 11 builds ML-KEM.

The canonical motivating scenario is two parties who have never met and want a fresh symmetric key, say a 256-bit key for the rest of the session. Alice and Bob can see each other’s public values over an authenticated channel but share no prior secret. The 1976 paper of Diffie and Hellman gave the first published solution to this problem (Diffie & Hellman, 1976). Fix a prime pp and a generator gg of the multiplicative group (Z/pZ)×(\mathbb{Z}/p\mathbb{Z})^\times. Alice picks a random exponent aa and sends A=ga mod pA = g^a \bmod p. Bob picks a random exponent bb and sends B=gb mod pB = g^b \bmod p. Each side then computes the shared value from the other side’s public value:

KA=Ba mod p,KB=Ab mod p.K_A = B^a \bmod p, \qquad K_B = A^b \bmod p.

Substituting A=gaA = g^a and B=gbB = g^b gives KA=gab=KBK_A = g^{ab} = K_B, so both sides derive the same element of the group. The whole exchange runs on a toy prime in a few lines of Python:

# Toy Diffie-Hellman in (Z/pZ)^* with p = 23 and g = 5.
# g is a primitive root mod 23, so ord(g) = phi(23) = 22.
p = 23
g = 5
# Alice picks a secret exponent and publishes A.
a = 6
A = pow(g, a, p)
# Bob picks a secret exponent and publishes B.
b = 15
B = pow(g, b, p)
# Each party computes the shared value from the other's public value.
alice_shared = pow(B, a, p)
bob_shared = pow(A, b, p)
print(A, B)
print(alice_shared == bob_shared, alice_shared)
# ==> 8 19
# ==> True 2

Every Python block this chapter prints is also a standalone file in the companion repository, under chapter-code/ch05/, one file per block. Appendix C covers the clone and the environment they run on.

The shared value 22 is not yet a usable AES key, since a group element is not a uniform byte string, so real deployments feed it through a key-derivation function (for example HKDF-Extract-then-Expand with SHA-256) to produce the symmetric key.

Now flip the question. Suppose Alice already has a 256-bit key KK that she chose herself, and she wants to send it to Bob. Diffie-Hellman cannot do this. Alice has no way to force the shared output to equal a specific KK she chose. Public-key encryption can. Alice encrypts KK under Bob’s public key and sends the ciphertext. But if Alice only wants Bob to end up with any fresh random key that Alice also knows, public-key encryption is overkill. Alice does not need to choose KK; she only needs some random KK that Bob can recover. That is the KEM problem, and it is the primitive the rest of the chapter develops.

A two-party key-agreement protocol is a pair of interactive algorithms (A,B)(A, B). Alice runs AA and Bob runs BB. Each party sends one or more public messages, and at the end both parties output a shared secret KK of some fixed bit length ℓK\ell_K. For Diffie-Hellman the two messages are A=gaA = g^a and B=gbB = g^b, and both parties output K=KDF(gab)K = \mathrm{KDF}(g^{ab}) where KDF\mathrm{KDF} is a key-derivation function. Security is stated against an adversary that reads both public messages but cannot actively interfere. Without an authenticated channel the scheme is vulnerable to man-in-the-middle, which a later section walks.

The post-quantum version of this story does not replace the math of Diffie-Hellman directly. Shor’s algorithm breaks the discrete logarithm in (Z/pZ)×(\mathbb{Z}/p\mathbb{Z})^\times and on elliptic curves (Shor, 1994), so every group that worked for pre-quantum Diffie-Hellman fails. Deployed post-quantum cryptography replaces classical Diffie-Hellman with a KEM rather than another interactive key-agreement protocol, though early lattice proposals (NewHope, Frodo) did start from the interactive framing.

A public-key encryption scheme is a triple (Gen,Enc,Dec)(\text{Gen}, \text{Enc}, \text{Dec}). Gen() samples a keypair. Enc(pk, m) produces a ciphertext cc. Dec(sk, c) returns the plaintext mm or a rejection symbol. Correctness asks that decrypting an honest encryption of mm return mm, except with small probability. Textbook RSA from Chapter 4 is the deterministic function m↦me mod nm \mapsto m^e \bmod n.

A KEM is a triple (Gen,Encap,Decap)(\text{Gen}, \text{Encap}, \text{Decap}). Gen() samples a keypair (pk,sk)(\text{pk}, \text{sk}). Encap(pk) uses fresh randomness to produce a ciphertext cc and a shared key K∈{0,1}ℓKK \in \{0, 1\}^{\ell_K}, and returns the pair (c,K)(c, K). The ciphertext cc travels over the wire, and both parties use KK as the shared symmetric key. Decap(sk, c) returns KK. What it returns for a ciphertext it rejects differs between schemes, and the Fujisaki-Okamoto section below takes that up. The conceptual split between a KEM and a PKE is exactly this: the caller of Encap supplies no key, and the encapsulation routine derives KK from fresh randomness. In a PKE, the sender chooses the plaintext. Boneh and Shoup give the abstract KEM/DEM decomposition and its security game (Boneh & Shoup, 2023).

The KEM/DEM flow Two columns labeled Alice and Bob, separated by a dashed wire. In Alice's column a box labeled encap of pk outputs a ciphertext ct and a symmetric key K. The ciphertext ct flows right across the wire into Bob's column. In Bob's column a box labeled decap of sk ct receives ct and outputs the same symmetric key K. Below both columns a second row shows K feeding a symmetric cipher on each side. Alice runs DEM dot Enc of K and the message, producing a symmetric ciphertext. The symmetric ciphertext flows right across a second wire into Bob. Bob runs DEM dot Dec of K and the symmetric ciphertext, recovering the original message. Alice Bob encap(pk) outputs (ct, K) ct decap(sk, ct) outputs K K (shared) K (shared) DEM.Enc(K, msg) outputs sym-ct sym-ct DEM.Dec(K, sym-ct) recovers msg KEM handles key establishment; DEM handles message confidentiality.
Figure 5.1. The KEM/DEM flow. The KEM handles key establishment: encap(pk) on Alice's side produces (ct, K), and Bob's decap(sk, ct) recovers the same K. NIST SP 800-227 notes that a KEM can be viewed as key transport or as key agreement depending on its construction, and places ML-KEM on the key-agreement reading, so "establishment" is the safer word for the general picture (National Institute of Standards and Technology, 2025b). The DEM handles message confidentiality with K. In a direct KEM/DEM construction the DEM is an authenticated encryption with associated data (AEAD) construction such as AES-GCM or ChaCha20-Poly1305, so a tampered symmetric ciphertext fails its integrity check. In a protocol such as TLS 1.3 there is no standalone DEM: K feeds a key schedule that derives the AEAD traffic keys, and the AEAD rather than the schedule is what rejects tampering.

KEM correctness asks that the legitimate decapsulation almost always return the same key that encapsulation produced. A KEM is ε\varepsilon-correct if

Pr⁡(pk,sk)←Gen(c,K)←Encap(pk)[ Decap(sk,c)≠K ]  ≤  ε,\Pr_{\substack{(\text{pk},\, \text{sk}) \,\gets\, \text{Gen} \\ (c,\, K) \,\gets\, \text{Encap}(\text{pk})}}\bigl[\, \text{Decap}(\text{sk}, c) \neq K \,\bigr] \;\leq\; \varepsilon,

where ε\varepsilon is a negligible function of the security parameter. Note which randomness the probability runs over: the key generation as well as the encapsulation. That is weaker than a per-key guarantee, and deliberately so: the published average does not imply a bound for every generated keypair, so quantifying over every honestly generated keypair would claim more than the figures below support. For lattice-based KEMs like ML-KEM the error ε\varepsilon is not exactly zero, because the underlying LWE decryption can fail on rare noise samples. FIPS 203 reports decapsulation-failure estimates of approximately 2−138.82^{-138.8} for ML-KEM-512, 2−164.82^{-164.8} for ML-KEM-768, and 2−174.82^{-174.8} for ML-KEM-1024 (National Institute of Standards and Technology, 2024). Chapter 11 develops the centered-binomial noise budget these rates come out of, and takes the exact figures from the same FIPS 203 analysis rather than re-deriving them.

Textbook RSA can be wrapped as a toy KEM. Encapsulation samples a random KK in [1,n−1][1, n - 1] and sets the ciphertext to c=Ke mod nc = K^e \bmod n. Decapsulation is K=cd mod nK = c^d \bmod n, which is the same computation as textbook RSA decryption. The snippet below builds the toy KEM on the 64-bit modulus from Chapter 4 and verifies a single round-trip:

# Toy RSA-KEM built on the 64-bit textbook RSA from Chapter 4.
# Encap samples a random K in [1, n - 1] and encrypts it as K^e mod n.
# Decap runs raw RSA decryption to recover the same K.
# Demo only: random.Random is deterministic and NOT cryptographically secure.
# Real encapsulation uses secrets.SystemRandom and feeds the output through
# a KDF to produce a fixed-length symmetric key.
import random
p = 3184935163
q = 3199286161
n = p * q
e = 65537
d = pow(e, -1, (p - 1) * (q - 1))
public_key = (n, e)
private_key = (n, d)
def encap(pk, rng):
mod, exp = pk
K = rng.randint(1, mod - 1)
c = pow(K, exp, mod)
return (c, K)
def decap(sk, c):
mod, dec_exp = sk
return pow(c, dec_exp, mod)
rng = random.Random(7) # fixed seed for reproducibility
c, K_alice = encap(public_key, rng)
K_bob = decap(private_key, c)
print(K_alice == K_bob)
print(0 < K_bob < n)
# ==> True
# ==> True

The round trip shows the toy is correct: decapsulation recovers the key that encapsulation chose. It is not secure, and breaking it takes only the public key. The question a KEM must answer well is a real-or-random one: shown a ciphertext cc and a key K′K', can an outsider tell whether K′K' is the key inside cc or an unrelated random key? Against the toy it can. It computes K′e mod nK'^e \bmod n and compares the result with cc. The two match when K′K' is the encapsulated key, and an unrelated random key matches with probability about 1/n1/n. Correctness constrains what the honest parties compute and says nothing about what an attacker can learn.

The toy also gives its key away to one decapsulation query. Textbook RSA is malleable: decapsulating the different ciphertext c⋅re mod nc \cdot r^e \bmod n returns rK mod nrK \bmod n, and multiplying by r−1 mod nr^{-1} \bmod n recovers KK. Exercise 7 works that attack, including the care it needs in choosing rr, but the comparison above needs no query at all.

ElGamal encryption is Diffie-Hellman used to send a message. Bob’s public key is a Diffie-Hellman share h=gxh = g^x. To encrypt a group element mm, Alice makes a fresh share grg^r and multiplies mm by the shared value hrh^r, which Bob can recompute as (gr)x(g^r)^x and divide out. The toy below works in the subgroup of squares modulo p=2063p = 2063, whose order q=1031q = 1031 is prime. Its messages must be squares, so 9=329 = 3^2 qualifies. A group this small has no security, since a laptop takes discrete logarithms in it instantly. It shows the mechanism only.

# Toy ElGamal: Diffie-Hellman used to encrypt.
# p = 2063 = 2 * 1031 + 1, and g = 4 generates the subgroup of squares,
# which has prime order q = 1031. Messages are squares mod p, so 9 qualifies.
import random
p, q, g = 2063, 1031, 4
rng = random.Random(5) # fixed seed for a reproducible page; keys come from secrets
x = rng.randrange(1, q) # Bob's private key
h = pow(g, x, p) # Bob's public key, a Diffie-Hellman share
def enc(h, m, r):
# Alice's fresh share g^r, and m masked by the shared value h^r.
return (pow(g, r, p), m * pow(h, r, p) % p)
def dec(x, c):
c1, c2 = c
shared = pow(c1, x, p) # Bob recomputes h^r as c1^x
return c2 * pow(shared, -1, p) % p
m = 9
c_first = enc(h, m, rng.randrange(1, q))
c_second = enc(h, m, rng.randrange(1, q))
print(c_first, c_second, dec(x, c_first) == dec(x, c_second) == m)
# The attacker holds a ciphertext c it cannot read, and can get any OTHER
# ciphertext decrypted. It multiplies the hidden message by 4 and asks.
c = enc(h, m, rng.randrange(1, q))
c_mauled = (c[0], c[1] * 4 % p)
reply = dec(x, c_mauled) # one decryption of a different ciphertext
print(c_mauled != c, reply * pow(4, -1, p) % p == m)
# ==> (1664, 1910) (1803, 412) True
# ==> True True

The first line shows what textbook RSA lacks. The same message encrypts to two different ciphertexts because each encryption draws a fresh rr, so the test that breaks textbook RSA, encrypting a guess and comparing, fails here. Defeating one test does not make a scheme secure. Against a classical eavesdropper, ElGamal’s security rests on an assumption. In a group where the decisional Diffie-Hellman (DDH) assumption holds, an efficient eavesdropper who picks two messages and sees the encryption of one cannot tell which it was noticeably better than by guessing (Exercise 11.5(a) in Boneh & Shoup, 2023). Shor’s algorithm breaks the scheme on a quantum computer by computing discrete logarithms (Chapter 4), and nothing in this chapter changes that.

The second line shows what ElGamal still lacks. The attacker holds a ciphertext cc it cannot read. It is not allowed to have cc itself decrypted, but (c1,4c2)(c_1, 4c_2) is a different ciphertext, and its decryption is 4m4m. Dividing by 4 recovers mm.

The attack needed three things: Bob’s public key, one ciphertext it wanted to read, and one decryption of a different ciphertext. It never touched the private key or a discrete logarithm. Real receivers have given away less than a decryption and still been broken. In 1998 Bleichenbacher showed that an RSA-encrypted SSL 3.0 key-exchange message can be decrypted by anyone who can learn, for ciphertexts of their choosing, one bit: whether the receiver finds the padding valid (Bleichenbacher, 1998). His experiments, run against an oracle he implemented, needed between 300,000 and 2 million queries. Of three SSL servers checked at the time, two gave the bit away through their error alerts. The next section writes these capabilities down as games.

Each game is played between a challenger, who generates the keypair (pk,sk)(\text{pk}, \text{sk}) and flips a uniform bit bb, and an attacker, who ends by outputting a guess b′b'. The two attack models name the games: indistinguishability under chosen-plaintext attack (IND-CPA) and under adaptive chosen-ciphertext attack (IND-CCA2), where adaptive means the attacker’s queries may continue after it has seen the challenge.

GameWhat the attacker getsWhat it must do
IND-CPA, PKEThe public key, and c∗=Enc(pk,mb)c^* = \mathrm{Enc}(\mathrm{pk}, m_b) for two equal-length messages m0,m1m_0, m_1 it choseGuess bb
IND-CCA2, PKEThe same, plus the decryption of any ciphertext other than c∗c^*, before and after the challengeGuess bb
IND-CPA, KEMThe public key, c∗c^*, and a key KbK_b that is the key encapsulated in c∗c^* (b=0b = 0) or a fresh uniform key (b=1b = 1)Guess bb
IND-CCA2, KEMThe same, plus the decapsulation of any ciphertext other than c∗c^*Guess bb

The eavesdropper’s game is called a chosen-plaintext attack because the public key already lets the attacker encrypt anything it chooses, so it needs no oracle. The decryption oracle stands for a receiver that can be made to process ciphertexts the attacker supplies: “an active adversary may be able to convince an honest user to decapsulate some ciphertexts of the adversary’s choosing” (National Institute of Standards and Technology, 2025b, sec. 2.3). The oracle returns the whole decryption, so a scheme that survives it also survives any answer a receiver computes from the decryption’s output and public information, such as an error message reporting ⊥\perp. Leaks of secret intermediate values, through timing for example, are outside the game, and the FIPS 203 paragraph below returns to them.

The only query the game excludes is c∗c^*, because answering it would let the attacker win against every scheme by asking for c∗c^* and comparing (National Institute of Standards and Technology, 2025b, sec. 2.3). Every other ciphertext may be submitted, including one derived from c∗c^*, which is exactly what the ElGamal attack submitted. The scheme’s answer to such a query may be a rejection. Real receivers keep no list of forbidden ciphertexts: the exclusion only stops the game from counting a trivial win.

The two KEM games choose no messages. The attacker gets c∗c^* together with a key that is either the one inside c∗c^* or a fresh random one, and must say which. The attack on a KEM is an attack on a key.

In the IND-CCA2 game for PKE the attacker’s advantage is

AdvIND-CCA2(A)=∣Pr⁡[b′=b]−1/2∣.\mathrm{Adv}^{\text{IND-CCA2}}(\mathcal{A}) = \bigl| \Pr[b' = b] - 1/2 \bigr|.

A public-key encryption scheme is IND-CCA2-secure if every polynomial-time adversary’s advantage is negligible in the security parameter (Boneh & Shoup, 2023). The other three rows define their advantages and their notions the same way. The prerequisites page says what makes a function of the security parameter negligible.

Replayed as games, the attacks above are wins for the attacker. Against textbook RSA it wins the IND-CPA game for PKE without a query: encrypt m0m_0 and compare with c∗c^*. A win that needs no query is also an IND-CCA2 win, since that game only adds an oracle the attacker need not use. Against the toy RSA-KEM it wins the KEM game without a query: compute Kbe mod nK_b^e \bmod n and compare with c∗c^*. Toy ElGamal is a PKE, the encrypt-and-compare test fails against it, and one decryption query wins the IND-CCA2 game. A scheme that withstands the first attack model and fails the second is what cryptographers mean when they call IND-CPA security weaker than IND-CCA2 security: weaker in what the attacker is allowed to do. Key length does not enter into it, and the ElGamal attack works at any size.

The Fujisaki-Okamoto transform: from PKE component to CCA-secure KEM

Section titled “The Fujisaki-Okamoto transform: from PKE component to CCA-secure KEM”

Bob cannot catch (c1,4c2)(c_1, 4c_2) by inspecting its plaintext. Every group element is a valid ElGamal message, so 4m4m looks as legitimate as mm. He could catch it by checking whether the ciphertext is the one an honest sender would have produced for that plaintext, but he cannot recompute that ciphertext, because the sender’s rr never reaches him. The Fujisaki-Okamoto transform removes the obstacle by deriving the randomness from the message, r=G(m)r = G(m) for a hash function GG. A ciphertext is then a fixed function of its plaintext and the public key, so Bob can decrypt, re-encrypt, and compare.

The transform also hashes the message into the shared key, so a related message yields an unrelated key. Deriving rr from mm makes encryption deterministic, which would let an attacker test guesses at a chosen message. In a KEM the message is a fresh random value from a set too large to search at real parameters, so guessing it is infeasible, which is one reason the modern transforms output a KEM.

The toy below applies the pattern to the ElGamal above. Encapsulation draws a random group element mm, encrypts it with randomness G(m)G(m), and outputs the key H(m,c)H(m, c). Decapsulation decrypts, re-encrypts with G(m′)G(m'), and compares. On a mismatch it does not report failure. It returns H(z,c)H(z, c), computed from a secret group element zz that Bob draws like a message and stores beside his private key, so the caller receives a key-shaped value either way.

# The same toy ElGamal, put through the Fujisaki-Okamoto pattern.
import hashlib
import random
p, q, g = 2063, 1031, 4
rng = random.Random(5) # fixed seed for a reproducible page; keys come from secrets
x = rng.randrange(1, q) # Bob's private key
h = pow(g, x, p) # Bob's public key
z = pow(g, rng.randrange(q), p) # Bob's rejection seed, a secret group element
def enc(h, m, r):
return (pow(g, r, p), m * pow(h, r, p) % p)
def dec(x, c):
c1, c2 = c
return c2 * pow(pow(c1, x, p), -1, p) % p
def hash_to_int(tag, *parts):
data = tag.encode() + b"|" + b"|".join(str(v).encode() for v in parts)
return int.from_bytes(hashlib.sha256(data).digest(), "big")
def G(m): # coins derived from the message
return hash_to_int("G", m) % q
def H(*parts): # the shared key, truncated to 32 bits for the page
return hash_to_int("H", *parts) % 2**32
def encap(h):
m = pow(g, rng.randrange(q), p) # a random message, never sent in clear
c = enc(h, m, G(m)) # coins come from G(m), not from rng
return c, H(m, c)
# No input checks: decap expects two subgroup elements, and c1 = 0 raises.
def decap(x, z, h, c):
m_prime = dec(x, c)
if enc(h, m_prime, G(m_prime)) == c: # re-encrypt and compare
return H(m_prime, c)
return H(z, c) # implicit rejection
c, K = encap(h)
print(decap(x, z, h, c) == K)
# The mauling from before. Decryption gives 4m, but the coins G(4m) do not
# reproduce the first component, so the comparison fails.
c_mauled = (c[0], c[1] * 4 % p)
m_prime = dec(x, c_mauled)
print(enc(h, m_prime, G(m_prime)) == c_mauled)
# What the attacker receives: a value fixed by (z, c_mauled), not by K.
k1 = decap(x, z, h, c_mauled)
k2 = decap(x, z, h, c_mauled)
k3 = decap(x, z, h, (c[0], c[1] * 9 % p))
print(k1 == k2, k1 != K, k1 != k3)
# ==> True
# ==> False
# ==> True True True

The first line is the honest round trip. The second is the check rejecting the mauled ciphertext: it decrypts to 4m4m, and re-encrypting 4m4m with randomness G(4m)G(4m) gives a different first component, because G(4m)G(4m) and G(m)G(m) differ. The check tests consistency, not whether the sender knew mm. A ciphertext passes exactly when it is the encryption of its own decryption under the derived randomness, so a mauled ciphertext whose two exponents happened to agree would pass. The theorem below bounds the attacker’s advantage over everything it might submit, lucky queries included.

The third line is what the attacker receives instead. The value is the same on every query with that ciphertext, because it is computed from zz and the ciphertext and from nothing random. It is unrelated to the challenge key. Because the ciphertext is an input, different rejected ciphertexts get fallbacks that behave like independent random keys rather than one shared constant, and like independent keys they can collide. One fixed fallback would be recognizable once seen.

The toy shows the mechanism and proves nothing about security. Its message set has 1031 elements, so an attacker can guess mm outright, which is the qRO/∣M∣q_{\mathrm{RO}}/|\mathcal{M}| term in the theorem below. If GG were an ideal random function, a given mauled ciphertext would pass the check with probability 1/10311/1031, the chance that G(4m)G(4m) equals G(m)G(m). The 1031 there counts the qq values GG can output, not the messages, although in ElGamal the two numbers are equal. ML-KEM’s messages are 256-bit strings.

A ciphertext that passes is an honest encryption of some message under Bob’s public key. Anyone holding that key can make one by running encapsulation, so passing says nothing about who sent it. The transform does not need every ciphertext an attacker makes to fail. It needs every accepted ciphertext to yield a key derived from that ciphertext’s own plaintext, which tells the attacker nothing about the challenge key. Authenticating the sender is a different job, and Chapter 6 gives it to signatures.

Hofheinz, Hövelmanns and Kiltz (HHK) analyze this pattern as one of a family of variants (Hofheinz et al., 2017, sec. 3). In their notation the toy’s pattern is FO⊥̸\mathrm{FO}^{\not\perp}, with implicit rejection, and the explicit-rejection variant FO⊥\mathrm{FO}^{\perp} returns a distinguished symbol ⊥\perp on a failed check instead.

The chapter states the theorem for the implicit-rejection variant, since that is the form ML-KEM relies on, and does not prove it.

Theorem (FO⊥̸\mathrm{FO}^{\not\perp}, informal). Let Π=(Gen,Enc,Dec)\Pi = (\mathrm{Gen}, \mathrm{Enc}, \mathrm{Dec}) be a public-key encryption scheme with message space M\mathcal{M}. Assume Π\Pi is IND-CPA-secure and δ\delta-correct, where δ\delta bounds the expectation over key generation of the worst-case message decryption-failure probability, so a scheme cannot be called δ\delta-correct on the strength of an average over mm. Let GG and HH be two hash functions modeled as random oracles. Define a KEM FO⊥̸(Π)\mathrm{FO}^{\not\perp}(\Pi) as follows. Key generation runs Π.Gen\Pi.\mathrm{Gen} and samples a uniform private seed z←Mz \gets \mathcal{M}, drawn from the same message space the construction encapsulates over. Encapsulation samples a random m∈Mm \in \mathcal{M}, computes c=Enc(pk,m; G(m))c = \mathrm{Enc}(\mathrm{pk}, m;\, G(m)) (the encryption coins are derived from G(m)G(m)), and returns (c,K)(c, K) with K=H(m,c)K = H(m, c). Decapsulation decrypts cc to m′m' and returns the implicit-rejection fallback Kˉ=H(z,c)\bar{K} = H(z, c) if Dec\mathrm{Dec} output ⊥\perp or if the re-encryption Enc(pk,m′; G(m′))\mathrm{Enc}(\mathrm{pk}, m';\, G(m')) differs from cc; otherwise it returns K=H(m′,c)K = H(m', c). Both failure conditions route to the fallback, and omitting the ⊥\perp case leaves the transform unsound. Then FO⊥̸(Π)\mathrm{FO}^{\not\perp}(\Pi) is IND-CCA2-secure in the random-oracle model, with

AdvIND-CCA2  ≤  qRO δ  +  3 qRO∣M∣  +  3 AdvIND-CPA,\mathrm{Adv}^{\text{IND-CCA2}} \;\leq\; q_{\mathrm{RO}}\, \delta \;+\; \frac{3\, q_{\mathrm{RO}}}{|\mathcal{M}|} \;+\; 3\, \mathrm{Adv}^{\text{IND-CPA}},

where qRO=qG+qHq_{\mathrm{RO}} = q_G + q_H counts the adversary’s random-oracle queries (Hofheinz et al., 2017).

Two things about that bound are worth reading off. The qRO/∣M∣q_{\mathrm{RO}} / |\mathcal{M}| term is why the theorem needs a large message space: an adversary that can guess mm wins outright, so ∣M∣|\mathcal{M}| must be big enough to make that term negligible. And the qRO δq_{\mathrm{RO}}\, \delta term is why δ\delta-correctness has to hold in the worst case over messages rather than on average: a chosen-ciphertext adversary picks which messages to push through the failure branch.

Returning H(z,c)H(z, c) instead of an error is implicit rejection. It is not what makes the transform secure. The re-encryption check and the hashing do that, and the explicit-rejection variant, which returns a distinguished symbol ⊥\perp, also has a proof in the random-oracle model under an extra condition on the encryption scheme (Hofheinz et al., 2017). What implicit rejection changes is the interface. A caller receives a key-shaped value either way, so a protocol built on the KEM has no rejection signal to branch on or to leak, and that signal is what Bleichenbacher’s attack read.

ML-KEM applies this pattern to an encryption scheme that FIPS 203 calls K-PKE, built on Module-LWE: Chapter 9 develops the problem and Chapter 11 builds the scheme. Three details differ from the toy. First, the success-path key and the encryption randomness come from one hash, (K,r)=G(m ∥ H(ek))(K, r) = G(m \,\|\, H(\mathrm{ek})), so the key depends on the message and the encapsulation key but not on the ciphertext.

Second, a failed check returns J(z ∥ c)J(z \,\|\, c), 32 bytes computed from a secret seed zz stored in the decapsulation key and from the ciphertext. The value is the same every time for a given key and ciphertext, and the caller receives 32 bytes either way, with no flag. FIPS 203 calls the comparison result the “implicit reject” flag, treats it as secret intermediate data, and states that “returning the value of the flag as an output in any form is not permitted” (National Institute of Standards and Technology, 2024, sec. 6.3). Implementations still have to compute the check and the conditional assignment without leaking the secret reject flag through timing, cache, or other side channels. NIST addresses that in SP 800-227 rather than in FIPS 203: cryptographic modules for KEMs “should be designed with appropriate countermeasures against side-channel attacks”, constant-time implementations among them (National Institute of Standards and Technology, 2025b, sec. 3.3).

Third, input checking is a separate step that runs before any of this. FIPS 203 §7.3 lists three checks: the ciphertext has the parameter set’s length, the decapsulation key has its length, and the encapsulation key embedded in the decapsulation key hashes to the value stored beside it. ML-KEM.Decaps “shall not be run” on inputs that have not been checked. FIPS 203 does not say how a failed check is reported, so what the caller sees then is the implementation’s choice. The ciphertext check runs on every call. The key checks need not, because FIPS 203 lets assurance that they were done come from other means (National Institute of Standards and Technology, 2024, sec. 7.3). Input checking passes any ciphertext of the right length, whoever made it. Once the checks have passed, a ciphertext that fails re-encryption produces the implicit-rejection key.

Textbook Diffie-Hellman falls to a man-in-the-middle without authentication. Against a purely passive eavesdropper it holds up: seeing A=gaA = g^a and B=gbB = g^b, that adversary cannot compute gabg^{ab} under the computational Diffie-Hellman (CDH) assumption. Concluding that the derived key KDF(gab)\mathrm{KDF}(g^{ab}) also looks random to it is a stronger statement needing a stronger assumption. That assumption is hash Diffie-Hellman, which CDH implies when the key-derivation function is modeled as a random oracle. DDH plus a suitable extractor is the other standard route (Boneh & Shoup, 2023).

An active adversary defeats the exchange without touching either assumption. Eve intercepts both messages, forwarding AE=gaEA_E = g^{a_E} to Bob in place of Alice’s AA, and BE=gbEB_E = g^{b_E} to Alice in place of Bob’s BB. Alice, who received BEB_E, computes BE a=gabEB_E^{\,a} = g^{a b_E}, which Eve recomputes as AbEA^{b_E}. Bob, who received AEA_E, computes AE b=gaEbA_E^{\,b} = g^{a_E b}, which Eve recomputes as BaEB^{a_E}. Each honest party pairs its own secret exponent with Eve’s matching public value, never with the other party’s, so Eve ends up holding two unrelated shared secrets while Alice and Bob each believe they share one with the other.

The defense is to authenticate the protocol, either via a signature on the exchange (as in TLS 1.3) or via a pre-shared long-term public key. Chapter 28 walks the TLS 1.3 handshake and where post-quantum key establishment lands in it; Chapter 29 covers the certificate chain that carries the authenticating signature, for a hierarchy the team owns.

Textbook RSA encryption has a fix of its own. Optimal asymmetric encryption padding (OAEP) randomizes the input before the RSA function sees it (Bellare & Rogaway, 1994), so the decryption of a mauled ciphertext is a random-looking block that almost never parses as a valid OAEP encoding. Real RSA-KEM is a different construction, and it is not OAEP. It encrypts a random integer zz with raw RSA, much as this chapter’s toy does, and derives the shared secret as KDF(z)\mathrm{KDF}(z) rather than using zz as the key, so no padding is involved and every zz is admissible (Housley & Turner, 2025). Testing a candidate key then needs zz, and recovering zz from the ciphertext means inverting RSA, so the comparison that broke the toy no longer works. RSAES-OAEP is a separate padded construction for encrypting a chosen message (Moriarty et al., 2016). The two are often conflated and should not be.

The three primitives are not interchangeable at deployment time.

PrimitiveWhat the receiver ends up withClassical examplePost-quantum status
Key agreementA shared secret both parties contributed randomness toDiffie-HellmanThe Ring-LWE variants in Chapter 9 are not widely deployed as standalone interactive protocols
Public-key encryptionThe specific message the sender choseRSA-OAEPPost-quantum standards expose KEM APIs rather than direct PKE APIs
KEMA random symmetric key that encapsulation generatesThe toy RSA-KEM aboveML-KEM (Chapter 11) is the finalized standard; HQC (Chapter 21) is selected but not yet published

Two rows of that last column need qualifying. Schemes such as ML-KEM and HQC do contain internal PKE-like components: K-PKE for ML-KEM, and the quasi-cyclic code-based PKE inside HQC. Those components are wrapped into KEMs before they are exposed to protocols, so a deployment never programs against the PKE API. ML-KEM’s standard is FIPS 203, published and final (National Institute of Standards and Technology, 2024). HQC was selected in March 2025 as a backup KEM for future standardization (National Institute of Standards and Technology, 2025a). No draft of its standard had been published by 19 October 2026, so it should not be described as a finalized FIPS standard.

IETF working groups are defining ML-KEM key establishment for TLS 1.3, SSH, and the Internet Key Exchange Protocol Version 2 (IKEv2) that sets up IPsec, and the three documents differ in both status and shape. The TLS one finished first: RFC 10024, Standards Track, specifies hybrid NamedGroups. The SSH one followed on 31 August 2026: RFC 10042, Informational, specifies hybrid key-exchange methods instead (Kampanakis et al., 2026; Kwiatkowski et al., 2026). The IKEv2 standards-track draft is the permissive one: it assigns the ML-KEM identifiers and allows ML-KEM alone as well as alongside another exchange, carried by the generic multiple-key-exchange framework of RFC 9370 (Kampanakis, 2026; Tjhai et al., 2023).

Deployment ran ahead of publication rather than following it: Go 1.24 enabled X25519MLKEM768 by default in crypto/tls while the TLS document was still a draft (The Go Authors, 2025; Valsorda, 2024). Chapters 27 and 28 walk the TLS 1.3 codepoints and rollout state in detail; neither covers SSH or IKEv2, whose documents are cited here and not developed further in this edition.

The post-quantum standardization effort concentrated on KEMs, not on PKE or interactive key agreement. The reason is that every practical protocol that used to need a key-agreement or PKE primitive (TLS, SSH, IPsec, hybrid public-key encryption stacks) can be rewired to use a KEM with a symmetric cipher on top. KEMs also fit hybrid encryption and one-pass key transport cleanly once the recipient’s encapsulation key is available. In interactive protocols such as TLS, SSH, or IKEv2 they are composed into the existing handshake rather than eliminating interaction altogether. “KEMs are cheaper” is therefore a context-dependent claim, not a universal one: ML-KEM ciphertexts are larger than X25519 key shares, for example. ML-KEM is the first post-quantum KEM standardized by NIST. Its security depends on the Module-LWE assumption Chapter 9 develops and on the Fujisaki-Okamoto transform above (Hofheinz et al., 2017; National Institute of Standards and Technology, 2024).

Where Chapter 5 ends and Chapter 6 picks up

Section titled “Where Chapter 5 ends and Chapter 6 picks up”

This chapter separated the three confidentiality primitives and put a security definition behind each one. Public-key encryption delivers the message the sender chose; key agreement delivers a secret both sides contributed randomness to; a KEM delivers a random key that encapsulation generates rather than one the sender supplies. IND-CCA2 is the bar for the first and the third, and the chapter showed two ways of missing it. Textbook RSA falls to one multiplication by rer^e, and a KEM built by naively wrapping a merely IND-CPA-secure PKE should not be assumed to clear the bar either. The Fujisaki-Okamoto transform is what closes that gap, and Chapter 11 instantiates it on the Module-LWE-based K-PKE to obtain ML-KEM.

Textbook Diffie-Hellman fails for a different reason, and that reason is the seam into Chapter 6. What breaks it is not the ciphertext-mauling attack that broke the toy KEM; it is the absence of authentication. The man-in-the-middle above breaks no computational assumption, it substitutes Eve’s own public values for Alice’s and Bob’s. Authentication is the signature’s job, and signatures carry their own security definition and their own textbook failures. Chapter 6 states that definition as the EUF-CMA game and forges a textbook RSA signature using the same multiplicative structure this chapter used to maul a ciphertext. It also pays off Chapter 4’s Exercise 4, recovering an ECDSA private key from two signatures that reused a nonce.

The two games share a skeleton. In both, the challenger holds the private key, and the attacker gets the public key and an oracle that runs the private-key operation on inputs of its choice. Both rule out a trivial win, in different ways: this chapter’s game excludes one decapsulation query, c∗c^*, while Chapter 6’s answers every signing query and counts a forgery only if its message was never submitted for signing. What changes is the goal. This chapter’s attacker tried to learn something hidden, while Chapter 6’s tries to produce something new: a valid signature on a message the signer was never asked to sign.

  1. Correctness is not security. The toy RSA-KEM decapsulates every ciphertext correctly. In the KEM game you receive the public key (n,e)(n, e), a ciphertext c∗c^* and a key KbK_b that is either the encapsulated key or a fresh random one. Win without asking for any decapsulation. Then say what this shows about correctness and security, and name the change real RSA-KEM makes that stops your attack.

  2. What the oracle stands for. In the IND-CCA2 game the attacker may decapsulate any ciphertext except c∗c^*. (a) Name two behaviors of a real receiver that this oracle stands for. (b) Why must the game exclude c∗c^* itself? (c) Why is it right that a ciphertext derived from c∗c^* is still a fair question?

  3. What “weaker” means. An attacker wins the IND-CPA game against textbook RSA encryption. The same attack fails against toy ElGamal, and one decryption query wins the IND-CCA2 game against it. (a) Give the IND-CPA attack on textbook RSA and say why it fails against ElGamal. (b) Give the IND-CCA2 attack on ElGamal. (c) State in what sense IND-CPA is weaker than IND-CCA2, and why a longer key would not turn an IND-CPA scheme into an IND-CCA2 one.

  4. Reproducible randomness and the fallback. Change the toy in the Fujisaki-Okamoto section so that a failed check returns rng.getrandbits(32) instead of H(z, c). Before running it, predict its three printed lines. Then answer two questions. (a) Why must the encryption randomness be derived from the message for the re-encryption check to mean anything? (b) Why is the fallback computed from a secret zz and from cc, rather than being a fixed dummy key or fresh random bytes on each call?

  5. The standard’s interface, and two claims to challenge. FIPS 203 requires an ML-KEM ciphertext to be 32(duk+dv)32(d_u k + d_v) bytes long (National Institute of Standards and Technology, 2024, sec. 7.3), and ML-KEM-768 has k=3k = 3, du=10d_u = 10 and dv=4d_v = 4 (Table 2 in National Institute of Standards and Technology, 2024). An ML-KEM-768 decapsulation receives two inputs: a ciphertext of 1,087 bytes, and a 1,088-byte string the attacker generated at random. (a) What does FIPS 203 require in each case, and what does the caller see? (b) A colleague says: “Decapsulation returned a key, so the ciphertext came from our partner, and ML-KEM is CCA-secure because of implicit rejection.” Correct both halves. (c) Name one way a deployment differs from the model in which the chapter’s theorem holds.

  6. Run the toy DH on a larger prime. Replace p=23p = 23 with p=2,063p = 2{,}063, which is prime, and keep g=5g = 5. Pick your own secret exponents aa and bb, run the same pow-based exchange, and confirm that alice_shared == bob_shared. Separately compute pow(5, 1031, 2063) and report whether the result is 11. How large does pp need to be before discrete logarithm becomes hard on a classical computer? Answer it from the Logjam analysis (Adrian et al., 2015) and NIST’s key-management guidance (Barker, 2020).

  7. Round-trip and maul the toy RSA-KEM. Take the toy RSA-KEM snippet above, run encap and decap, and confirm the round-trip. Then try a mauling attack. Take r=2r = 2, compute c′=(c⋅pow(r,e,n)) mod nc' = (c \cdot \mathrm{pow}(r, e, n)) \bmod n, and run decap(private_key, c_prime). Check that the result is (r⋅K) mod n(r \cdot K) \bmod n, then recover KK as (decap_output⋅pow(r,−1,n)) mod n(\mathrm{decap\_output} \cdot \mathrm{pow}(r, -1, n)) \bmod n.

    Now show why “any invertible r≠1r \neq 1” is not good enough. In the CCA game the challenge ciphertext is excluded from the oracle, so the attack needs c′≠cc' \neq c, and invertibility does not deliver that. Take K=pK = p and pick rr by the Chinese remainder theorem with r≡2(modp)r \equiv 2 \pmod p and r≡1(modq)r \equiv 1 \pmod q. Verify that this rr is invertible and is not 1 mod n1 \bmod n, then verify that c′=cc' = c exactly, so the query is forbidden. Then show that r=2r = 2 escapes this for every KK, and under every valid two-prime RSA key rather than only this one. The property your argument needs is what ee being invertible modulo (p−1)(q−1)(p-1)(q-1) forces about gcd⁡(e,p−1)\gcd(e, p-1) and gcd⁡(e,q−1)\gcd(e, q-1).

    Finally, two short answers. Say what extra restriction the same attack needs in the PKE game, where the adversary picks the challenge plaintexts instead of encapsulation sampling KK. Then explain in two sentences why the attack fails against an FO⊥̸\mathrm{FO}^{\not\perp}-transformed KEM (the implicit-rejection transform this chapter teaches, which ML-KEM uses in a modified form).

  8. Prove the toy KEM’s correctness, and bound the non-coprime fraction. First, show that decap(private_key, c) == K for every K produced by encap. The clean argument is by the Chinese remainder theorem. Fermat’s little theorem gives Ked≡K(modp)K^{ed} \equiv K \pmod{p} and Ked≡K(modq)K^{ed} \equiv K \pmod{q}, trivially so when p∣Kp \mid K or q∣Kq \mid K, and CRT lifts the pair to Ked≡K(modpq)K^{ed} \equiv K \pmod{pq}. Euler’s theorem from Chapter 4 covers only the coprime case; CRT closes the remaining residues.

    Second, bound the non-coprime fraction. Use inclusion-exclusion on the sets of pp-multiples and qq-multiples in [1,n−1][1, n-1]. Show that a uniform KK shares a factor with n=pqn = pq with probability (p+q−2)/(pq−1)<1/p+1/q(p + q - 2) / (pq - 1) < 1/p + 1/q, which for the 32-bit primes in the snippet is below 2−302^{-30}. Then say what this fraction does not govern. It is not a correctness bound, since the first part showed decap is exact on the whole range. Nor is it the mauling attack’s failure rate: that attack needs its blinding factor rr to be invertible, not KK, and it chooses rr deliberately. Which rr would you pick, for an RSA modulus, to make the choice trivially safe?

  9. State the FO⊥̸\mathrm{FO}^{\not\perp} theorem in your own words. Without copying from this chapter, state the inputs, the output, the role of the private seed zz, the behavior of decapsulation when the re-encryption check fails, the security game, and the assumption on the random oracles. The exercise is not to prove the theorem; it is to produce a statement precise enough that someone else could look up the proof in (Hofheinz et al., 2017).

Worked solutions and editorial notes for these exercises are in Appendix D, Chapter 5. A separate track, for rebuilding rather than reading: the package exercises/ch05-kem-primitives has every function the chapter teaches replaced by a stub. Run PQC_IMPL=exercises pytest tests/ch05 to grade your version against the suite that proves the reference one.

Adrian, D., Bhargavan, K., Durumeric, Z., Gaudry, P., Green, M., Halderman, J. A., Heninger, N., Springall, D., Thomé, E., Valenta, L., VanderSloot, B., Wustrow, E., Zanella-Béguelin, S., & Zimmermann, P. (2015). Imperfect Forward Secrecy: How Diffie-Hellman Fails in Practice. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS 2015), 5–17. https://doi.org/10.1145/2810103.2813707
Barker, E. (2020). Recommendation for Key Management, Part 1: General Revision 5. NIST Special Publication 800-57 Part 1 Revision 5. https://doi.org/10.6028/NIST.SP.800-57pt1r5
Bellare, M., & Rogaway, P. (1994). Optimal asymmetric encryption. Advances in Cryptology – EUROCRYPT 1994, 950, 92–111. https://doi.org/10.1007/BFb0053428
Bleichenbacher, D. (1998). Chosen ciphertext attacks against protocols based on the RSA encryption standard PKCS #1. Advances in Cryptology – CRYPTO ’98, 1462, 1–12. https://doi.org/10.1007/BFb0055716
Boneh, D., & Shoup, V. (2023). A Graduate Course in Applied Cryptography (v0.6). Free online textbook. https://toc.cryptobook.us/
Diffie, W., & Hellman, M. E. (1976). New directions in cryptography. IEEE Transactions on Information Theory, 22(6), 644–654. https://doi.org/10.1109/TIT.1976.1055638
Fujisaki, E., & Okamoto, T. (1999). Secure integration of asymmetric and symmetric encryption schemes. Advances in Cryptology – CRYPTO 1999, 1666, 537–554. https://doi.org/10.1007/3-540-48405-1_34
Fujisaki, E., & Okamoto, T. (2013). Secure integration of asymmetric and symmetric encryption schemes. Journal of Cryptology, 26(1), 80–101. https://doi.org/10.1007/s00145-011-9114-1
Fujisaki, E., Okamoto, T., Pointcheval, D., & Stern, J. (2001). RSA-OAEP is secure under the RSA assumption. Advances in Cryptology – CRYPTO 2001, 2139, 260–274. https://doi.org/10.1007/3-540-44647-8_16
Hofheinz, D., Hövelmanns, K., & Kiltz, E. (2017). A modular analysis of the Fujisaki-Okamoto transformation. Theory of Cryptography – TCC 2017, Part I, 10677, 341–371. https://doi.org/10.1007/978-3-319-70500-2_12
Housley, R., & Turner, S. (2025). Use of the RSA-KEM Algorithm in the Cryptographic Message Syntax (CMS). IETF RFC 9690. https://doi.org/10.17487/RFC9690
Kampanakis, P. (2026). Post-quantum Key Exchange with ML-KEM in the Internet Key Exchange Protocol Version 2 (IKEv2). IETF Internet-Draft, draft-ietf-ipsecme-ikev2-mlkem-09. https://datatracker.ietf.org/doc/draft-ietf-ipsecme-ikev2-mlkem/
Kampanakis, P., Stebila, D., & Hansen, T. (2026). Post-Quantum/Traditional Hybrid Key Exchange with the Module-Lattice-Based Key-Encapsulation Mechanism for Use in SSH. RFC 10042, Informational. https://www.rfc-editor.org/rfc/rfc10042.html
Kwiatkowski, K., Kampanakis, P., Westerbaan, B., & Stebila, D. (2026). Post-Quantum Traditional (PQ/T) Hybrid Key Agreement Mechanisms for TLS 1.3. RFC 10024. https://doi.org/10.17487/RFC10024
Moriarty, K., Kaliski, B., Jonsson, J., & Rusch, A. (2016). PKCS #1: RSA Cryptography Specifications Version 2.2. RFC 8017. https://doi.org/10.17487/RFC8017
National Institute of Standards and Technology. (2024). FIPS 203: Module-Lattice-Based Key-Encapsulation Mechanism Standard. Federal Information Processing Standards Publication. https://doi.org/10.6028/NIST.FIPS.203
National Institute of Standards and Technology. (2025a). NIST Selects HQC as Fifth Algorithm for Post-Quantum Encryption. NIST news release. https://www.nist.gov/news-events/news/2025/03/nist-selects-hqc-fifth-algorithm-post-quantum-encryption
National Institute of Standards and Technology. (2025b). Recommendations for Key-Encapsulation Mechanisms. NIST Special Publication 800-227. https://doi.org/10.6028/NIST.SP.800-227
Shor, P. W. (1994). Algorithms for quantum computation: discrete logarithms and factoring. Proceedings of the 35th Annual Symposium on Foundations of Computer Science (FOCS), 124–134. https://doi.org/10.1109/SFCS.1994.365700
Shoup, V. (2001). OAEP reconsidered. Advances in Cryptology – CRYPTO 2001, 2139, 239–259. https://doi.org/10.1007/3-540-44647-8_15
The Go Authors. (2025). Go 1.24 Release Notes: crypto/tls. go.dev/doc/go1.24. https://go.dev/doc/go1.24
Tjhai, C., Tomlinson, M., Bartlett, G., Fluhrer, S., Van Geest, D., Garcia-Morchon, O., & Smyslov, V. (2023). Multiple Key Exchanges in the Internet Key Exchange Protocol Version 2 (IKEv2). IETF RFC 9370. https://doi.org/10.17487/RFC9370
Valsorda, F. (2024). crypto/tls: add X25519MLKEM768 and use by default; remove x25519Kyber768Draft00. Go issue 69985. https://go.dev/issue/69985

Last updated: