How MTU and encapsulation overhead work

Every wrapper costs bytes. When the total exceeds what the path carries, the symptom is a connection that starts and then hangs.

Maximum Transmission Unit is the largest packet a link will carry. Ethernet's familiar 1500 bytes is the payload, not the frame — the 14-byte header sits outside it, which is the first thing that confuses people.

Where the bytes go

• MPLS label — 4 bytes each. A VPN stack is two, so 8.
• 802.1Q VLAN tag — 4 bytes. QinQ adds another 4.
• IPv4 header — 20 bytes. IPv6 — 40.
• GRE — 4 bytes, plus the outer IP header.
• VXLAN — 8 bytes, over UDP and IP, so about 50 in total.
• IPsec ESP — variable, and this is the one to be careful with.

Why the IPsec number is never exact

ESP overhead depends on the cipher, the key size, the padding needed to reach a block boundary, and whether NAT traversal wraps the whole thing in UDP. Any single figure is a planning estimate.

Tools that print one exact number for IPsec are a common reason tunnels get sized wrongly. Use a worst case, then confirm against the devices.

The symptom worth recognising

When a path cannot carry a full-sized packet, one of two things happens.

Fragmentation — the packet is split and reassembled. It works and costs performance.

Path MTU discovery — the sender is told to send smaller packets by an ICMP message. This is the designed behaviour, and it depends on ICMP getting through. A great many networks filter ICMP as a blanket security measure, and when those messages are dropped the sender never learns and keeps sending packets that vanish.

The result is the classic MTU symptom: the connection establishes, small exchanges work, and the first large transfer hangs. An SSH session that connects and then freezes when you run a command producing a lot of output is the canonical example. Anything that behaves like that is an MTU problem until proven otherwise.

What to do

• Raise the underlay MTU where you control it. Cores frequently run 9000 bytes so encapsulation never reaches the customer.
• Lower the inner MTU where you do not.
• Clamp TCP MSS on tunnel interfaces. It fixes TCP without relying on ICMP, which is why it is so widely deployed — though it does nothing for UDP.
• Do not filter all ICMP. Blocking fragmentation-needed messages breaks path MTU discovery for everyone behind you.

Frequently asked questions

Why does my connection work but large transfers hang?

Almost always MTU. A packet too large for the path is dropped, and the ICMP message that would tell the sender is filtered somewhere, so it never learns and keeps retrying.

How much does an MPLS label cost?

Four bytes each. A two-label L3VPN stack costs eight, so a 1500-byte core link carries 1492 bytes of customer packet.

What is MSS clamping?

Rewriting the maximum segment size in TCP handshakes so both ends agree to send smaller segments. It fixes TCP without depending on ICMP, and does nothing for UDP.

Should I just enable jumbo frames?

In a core you control, often yes. Every device on the path has to agree, and a single one left at 1500 produces exactly the intermittent failure you were trying to avoid.