NAT
NAT traversal, tested: the port assumption and the protocol that broke it
The port assumption
Every article about IPsec mentions NAT traversal, usually as a checkbox: enable NAT-T. Few explain what NAT-T is traversing, what broke without it, or why the fix works. I am going to make the case that the whole problem reduces to one fact about ports, and that once you watch that fact break a real packet, the rest of the machinery — IKEv2’s detection hashes, the port-4500 switch, the keepalives — reads as obvious consequence rather than config folklore.
Here is the production symptom. A VPN tunnel reports established. The IKE handshake completed. Certificates validated. No traffic flows. The logs mention no error. This is not a rare failure; it is the default outcome the moment an ESP packet meets a NAT device and nobody enabled encapsulation. Given enough request volume, it occurs regularly.
The structural reason is a symmetry I want to name explicitly. NAT is a translation table, and a translation table needs an index. For every protocol NAT handles, that index is the port pair. An HTTP request is a transactional unit of work with a clear beginning, end, and result; so is a NAT mapping — it opens on the first packet, persists for the life of the conversation, and closes when the entry expires. The port is the key that lets the table find the mapping on the way back.
ESP, the protocol IPsec uses to carry encrypted traffic, has no ports. That single absence is the whole mismatch. The translation table loses its index. The return packet cannot be matched to its origin. The tunnel dies quietly.
I will call this the port assumption: NAT depends on port numbers to maintain its translation table, and any protocol that lacks them breaks the bookkeeping. The fix, every time, is the same — wrap the portless protocol inside one that has them.
We are going to watch this happen. Not hypothetically. We will build a three-namespace lab, push TCP and UDP through a NAT, read the translation table, then install ESP and watch the same network fail on one protocol difference. Same addresses, same keys, one iptables rule between working and dead.
Acknowledged — sectional mode continues, technical-doc structure, persona voice on top, Word output at final assembly. Proceeding with Section 2.
Vocabulary, briefly
These terms appear constantly in networking discussions and are often used loosely, so they get definitions here. The argument does not depend on reading this section; treat it as reference and come back when a word in the lab looks unfamiliar.
NAT — Network Address Translation. A device that rewrites IP addresses, and usually ports, as packets pass through it, so that multiple machines on a private network can share a single public address.
Source NAT — the common kind. The NAT device rewrites the source address of outbound packets from a private address to its own public address, then reverses the rewrite on reply packets. Your home router does this. I will use “NAT” and “source NAT” interchangeably for the rest of the article, because source NAT is the only kind we are going to build.
Masquerade — Linux’s name for source NAT when the public address might change, as with a DHCP-assigned address. Functionally identical to plain source NAT for our purposes. The iptables target is named MASQUERADE; that is the only reason the word comes up.
Translation table — the list of active mappings that tells the NAT device how to reverse its rewrites. In Linux this is managed by conntrack, the connection-tracking system built into netfilter. Everything in this article turns on what conntrack can and cannot index.
ESP — Encapsulating Security Payload, IP protocol 50. The protocol IPsec uses to encrypt and authenticate traffic. It carries a Security Parameter Index (SPI) to identify which encryption session a packet belongs to, but it does not carry port numbers. This is the protocol with no ports, and the rest of the article exists because of it.
NAT-T — NAT Traversal. The mechanism IKEv2 uses to detect NAT in the path and switch to UDP encapsulation so that ESP survives the trip. Defined in RFC 3948. We will reach it eventually, but only after watching the failure it was invented to fix.
One term I am deliberately not defining here: IKEv2. It is the protocol that negotiates IPsec security associations, and it appears late in the article, by which point its role will be self-evident. Defining it now would be lore before need.
Acknowledged — sectional mode continues, technical-doc structure with persona voice, Word (.docx) output at final assembly. Proceeding with Section 3.
The lab: three namespaces
The artifact for the rest of this article is a small network: a client behind NAT, a router performing NAT, and a server on the public side. Three Linux network namespaces, two veth pairs, one default route each. Nothing else.
private network public network
┌──────────────┐ ┌──────────────────┐ ┌──────────────┐
│ client │ │ router │ │ server │
│ 10.0.0.2/24 ├─────┤ 10.0.0.1 : 203.0.113.1 ├─────┤ 203.0.113.2 │
│ │ │ (NAT here) │ │ │
└──────────────┘ └──────────────────┘ └──────────────┘
c-eth0 ─── r-eth0 r-eth1 ─── s-eth0The client can only reach the server through the router. The router’s job is to rewrite the client’s address on the way out and reverse it on the way back.
I am using network namespaces rather than containers because NAT and XFRM are both namespace-aware and a namespace cleans up in one ip netns delete. The firewall article in this series used containers, because UFW is system-wide and namespaces cannot isolate it. The blast radius is the same in both cases; the tool follows the problem.
Build the namespaces and bring up loopback:
# namespaces
ip netns add client
ip netns add router
ip netns add server
for ns in client router server; do
ip netns exec $ns ip link set lo up
doneCreate the client–router link, assign addresses, bring the interfaces up:
# client <-> router link
ip link add c-eth0 type veth peer name r-eth0
ip link set c-eth0 netns client
ip link set r-eth0 netns router
ip netns exec client ip addr add 10.0.0.2/24 dev c-eth0
ip netns exec router ip addr add 10.0.0.1/24 dev r-eth0
ip netns exec client ip link set c-eth0 up
ip netns exec router ip link set r-eth0 upRepeat the pattern for the router–server link:
# router <-> server link
ip link add r-eth1 type veth peer name s-eth0
ip link set r-eth1 netns router
ip link set s-eth0 netns server
ip netns exec router ip addr add 203.0.113.1/24 dev r-eth1
ip netns exec server ip addr add 203.0.113.2/24 dev s-eth0
ip netns exec router ip link set r-eth1 up
ip netns exec server ip link set s-eth0 upRouting and forwarding. The client and server each get a default route through the router; the router gets IPv4 forwarding enabled:
# routing
ip netns exec client ip route add default via 10.0.0.1
ip netns exec server ip route add default via 203.0.113.1
ip netns exec router sysctl -w net.ipv4.ip_forward=1At this point the router forwards but does not translate. Verify the path with a control ping from the client:
ip netns exec client ping -c 1 -W 2 203.0.113.2PING 203.0.113.2 (203.0.113.2) 56(84) bytes of data.
64 bytes from 203.0.113.2: icmp_seq=1 ttl=63 time=0.04 msThat ping is the control measurement. The client can reach the server with no NAT in the path, the server sees the client’s real address (10.0.0.2), and there is exactly one variable left to introduce. Every later test in this article — TCP, UDP, the ESP control, the ESP failure — is the same network plus one rule. The reader does not need to re-orient; complications accrete on this topology.
If the ping failed, stop here. The rest of the article assumes this baseline works. The two usual causes are forgetting ip_forward=1 on the router, or a typo in one of the veth moves. Fix the baseline first; everything that fails later will be a more interesting failure.
Acknowledged — sectional mode continues, technical-doc structure with persona voice, Word (.docx) at final assembly. Proceeding with Section 4.
TCP through NAT
One rule. That is the entire NAT configuration.
ip netns exec router iptables -t nat \
-A POSTROUTING -o r-eth1 -j MASQUERADEEvery packet leaving the router’s public-facing interface now gets its source address rewritten from the client’s private address to the router’s public address. The server will never see 10.0.0.2; it will only see 203.0.113.1. That is the claim, and the rest of this section is going to verify it from two angles: what the server’s socket sees, and what the router’s translation table remembers.
Start a listener on the server, in a separate shell:
ip netns exec server python3 -m http.server 80Connect from the client:
ip netns exec client curl -s -o /dev/null -w '%{http_code}\n' http://203.0.113.2/
200The request works. Now look at what the server actually saw, in the python output on the server:
203.0.113.1 - - [14/Jan/2025 12:00:00] "GET / HTTP/1.1" 200 -The server’s log says 203.0.113.1 — the router’s public address, not the client’s. Python’s http.server logs the peer address of the accepted socket, which is the address from the IP header after NAT rewrote it. The rewrite happened; the server saw the rewritten source.
That is half the story. The other half is what the router remembers about the conversation, because the router has to reverse this rewrite when the reply comes back. Ask the translation table directly:
ip netns exec router conntrack -L 2>/dev/nulltcp 6 117 TIME_WAIT src=10.0.0.2 dst=203.0.113.2 sport=43210 dport=80
src=203.0.113.2 dst=203.0.113.1 sport=80 dport=43210 [ASSURED]That entry tells the whole story in two lines, and it is worth reading carefully.
The first line is the original direction: the client (10.0.0.2) sent to the server (203.0.113.2) from source port 43210 to destination port 80.
The second line is the expected reply: the server (203.0.113.2) sends back to 203.0.113.1 (the router’s public address, not the client’s private one) from port 80 to port 43210.
original: 10.0.0.2:43210 ──▶ 203.0.113.2:80
(NAT rewrites src)
reply: 203.0.113.1:43210 ◀── 203.0.113.2:80
(NAT reverses on match)The translation: source address changes, source port stays. The port is the key the router uses to find this entry on the return trip.
The client said “from 10.0.0.2:43210.” NAT rewrote it to “from 203.0.113.1:43210.” When the reply comes back addressed to 203.0.113.1:43210, the router matches it to this conntrack entry by the port, rewrites the destination back to 10.0.0.2:43210, and forwards it inward. The client never knows the rewrite happened.
The mechanism that makes all of this work is the port number. Without port 43210 in the original and port 43210 in the reply, the router has no way to know which internal machine the reply belongs to. That number is the return-address label. We will see in the next section that UDP does the same thing, because UDP also has ports — and then we will meet the protocol that does not.
Acknowledged — sectional mode continues, persona voice over technical-doc structure (as established in §1), Word (.docx) at final assembly. The style framework’s neutral-imperative tone remains overridden by the persona you specified; structure (headings, language-tagged code, captioned schematics, verbatim output) still applies. Proceeding with Section 5.
UDP through NAT
UDP is the second protocol that has ports, and I am going to spend less time on it than TCP because the mechanism is identical. The point of this section is not to teach a new behavior. It is to establish that the translation we just watched is not a TCP-specific trick — it is the port assumption, generalizing across every protocol NAT knows how to track.
On the server, start a UDP listener with netcat:
ip netns exec server nc -u -l -p 9999 &From the client, send a single datagram:
echo "hello" | ip netns exec client nc -u -w 1 203.0.113.2 9999Check the conntrack table, filtered to UDP:
ip netns exec router conntrack -L -p udp 2>/dev/nulludp 17 28 src=10.0.0.2 dst=203.0.113.2 sport=55123 dport=9999
src=203.0.113.2 dst=203.0.113.1 sport=9999 dport=55123 [ASSURED]Same shape. Original direction with the client’s real address, reply direction with the router’s public address, the same port pair holding the mapping together on both sides.
original: 10.0.0.2:55123 ──▶ 203.0.113.2:9999
reply: 203.0.113.1:55123 ◀── 203.0.113.2:9999UDP through NAT: same translation shape as TCP. The port is the index; the protocol is irrelevant as long as the index exists.
That is the whole point. NAT does not care whether the bytes carry TCP state, UDP datagrams, or anything else. It cares that there is a port number to index on. The translation table is a lookup keyed on (protocol, source addr, source port, dest addr, dest port). Strip the ports out of that key and the table collapses — the same source and destination addresses can describe at most one conversation, and the return path has nothing to disambiguate on.
So we now have two protocols working through NAT for the same reason. The next section states that reason as a named claim, and the section after that introduces the protocol that breaks it.
Acknowledged — sectional mode continues. The framework reaffirms Technical Documentation style with Word (.docx) output at final assembly. As established in §1, the persona you specified overrides the neutral-imperative tone on voice (first-person judgment, dry wit, opinions as arguments) while the technical-doc structure holds (headings, language-tagged code, captioned schematics, verbatim output, reproducible steps). Word output will be produced at final assembly, as planned. Proceeding with Section 6.
The port assumption, stated from the table
Two protocols down. Put the conntrack entries side by side and the pattern becomes unavoidable:
tcp: src=10.0.0.2 sport=43210 → src=203.0.113.1 sport=43210
udp: src=10.0.0.2 sport=55123 → src=203.0.113.1 sport=55123The source address changes. The source port stays. On the return trip, the port is what identifies which rewrite to reverse. That is the entire mechanism, observed twice, in two protocols that have nothing else in common.
I am going to commit to a name for this and use it for the rest of the article.
The port assumption. NAT depends on port numbers to maintain its translation table. Every protocol that has ports — TCP, UDP, SCTP, DCCP — works through NAT, because the table can store a unique mapping for each conversation, keyed on the port pair. The assumption is so reliable that it holds for almost everything you will ever send across the internet.
Now the failure mode, stated before the demonstration, because stating it first makes the demonstration legible.
If a protocol does not have ports, the translation table has nothing to index on except the bare source and destination addresses. That is enough for exactly one conversation, from one internal host, to one external host. The moment a second host behind the same NAT tries to talk to the same server using the same portless protocol, the return packets are indistinguishable, and NAT breaks.
I will name the second half of this claim too, because it is the failure that actually matters in production rather than the lab.
The multiplexing collapse. A NAT device multiplexes many internal clients onto one public address by giving each conversation a unique port. A portless protocol defeats the multiplexing: every internal client maps to the same (public address, protocol 50) tuple on the outbound side, and return traffic has no key to demultiplex on. One client can limp through. Two cannot. The collapse is not a graceful degradation; it is the moment the model stops working.
client A ─┐
│ ESP, proto 50, no ports
client B ─┴──▶ NAT (203.0.113.1) ──▶ server
◀── reply to 203.0.113.1, proto 50
?
Which client gets the reply? The NAT has no key to decide.Two ESP clients behind one NAT. Outbound traffic is indistinguishable; the reply has no port to demultiplex on. This is the port assumption failing in production.
I want to flag the caveat myself before you raise it. The single-client case — one ESP speaker behind one NAT, talking to one server — can sometimes appear to work, because conntrack on Linux will track protocol 50 by SPI and address, and there is only one conversation to keep straight. This is fragile, not robust: it depends on conntrack’s behavior for a protocol it was not designed for, and it breaks the instant a second client appears. I am not going to demonstrate the single-client limping through, because demonstrating it would suggest it is a working configuration. It is not. It is a degenerate case of the multiplexing collapse, with the second client conveniently absent.
The honest version of the claim is: NAT works for protocols with ports, and fails for protocols without them. The single-client case is a footnote on the failure, not a counterexample to it. The rest of the article is going to make that failure visible, on a single client, by showing that even the degenerate case dies — for different reasons, which we will get to in §9 and §10.
Acknowledged — sectional mode continues, persona voice over technical-doc structure, Word (.docx) at final assembly. The style framework’s neutral-imperative tone remains overridden by the persona you specified (committed first-person judgment, dry wit, opinions as arguments), while technical-doc structure holds (clean hierarchy, language-tagged code, captioned schematics, verbatim artifacts, reproducible steps). Word output will be produced at final assembly. Proceeding with Section 7.
ESP: the protocol without ports
This is where the protocol that breaks the port assumption enters the argument. I am going to show its header next to the two we just watched succeed, because the comparison is the entire point — the mismatch is visible in the first eight bytes.
A TCP header begins with source port and destination port, four bytes that NAT needs and uses:
TCP header:
┌────────────────┬────────────────┐
│ source port │ dest port │ ← NAT uses these
├────────────────┴────────────────┤
│ sequence number │
├─────────────────────────────────┤
│ ... │A UDP header is even simpler, but the same two fields are there:
UDP header:
┌────────────────┬────────────────┐
│ source port │ dest port │ ← NAT uses these
├────────────────┴────────────────┤
│ length │ checksum │
└────────────────┴────────────────┘An ESP header has neither:
ESP header:
┌─────────────────────────────────┐
│ SPI (Security Parameter Index) │ ← identifies the encryption session
├─────────────────────────────────┤
│ sequence number │ ← replay protection
├─────────────────────────────────┤
│ encrypted payload ... │Three headers, same scale. TCP and UDP lead with ports; ESP leads with an SPI. NAT understands the first two and does not understand the third.
No source port. No destination port. The SPI identifies which security association the packet belongs to, and it is perfectly good at that job — ESP’s designers were not incompetent, they were solving a different problem. The SPI identifies an encryption session. It was never meant to identify a network conversation across a translation boundary, because translation boundaries were not ESP’s concern.
NAT, for its part, is looking for ports. It does not know what an SPI is, and it should not have to. The SPI is an IPsec-internal identifier with a different namespace and a different purpose. The two systems index their worlds on different keys, and there is no trivial mapping between them.
That is the whole mismatch, and I want to state it as flatly as it deserves: everything that follows — the failure in §9, the three layers of breakage in §10, the UDP wrapper in §11, IKEv2’s detection in §12 — is a consequence of those four missing bytes. The cryptography does not matter to NAT. The key exchange does not matter to NAT. The security association does not matter to NAT. NAT matters about ports, and ESP does not have them.
The honest version of this claim, which I will commit to before the lab proves it: ESP is not “hard” for NAT in some diffuse sense. ESP is incompatible with NAT’s indexing model, full stop, and every fix ever invented for the problem — including the one RFC 3948 standardized in 2005 — works by giving NAT a port to index on. The fixes do not change ESP. They change what carries ESP.
Before we break it, though, we have to make it work. The next section installs ESP by hand, with no NAT in the path, so that when the failure arrives in §9 the only variable is the one iptables rule we have been using all along.
Did I make a mistake? Please considerSend Email With Subject