IP Fragmentation and Reassembly, More Than Just a Textbook Topic

IP fragmentation and reassembly. An old topic. Textbooks explain it cleanly: routers split packets when the next hop MTU is smaller, and receivers stitch fragments back together using offsets. But anyone who has run real traffic knows the truth is messier.

This article walks through the fundamentals, the Linux kernel knobs, how a userspace stack like mipstack handles fragments, and what Cisco’s VFR mechanism actually brings to SD-WAN. No fluff, just practical stuff.

The Basics First, IPv4 and IPv6 Disagree Deeply

IPv4 puts fragmentation capability at the network layer. Routers can split packets. The original rationale is simple: physical networks differ a lot, from 1500-byte Ethernet to 576-byte dial-up links. Expecting every sender to probe the path would be unrealistic, so IPv4 routers take on the job. Hosts barely notice what happened.

IPv6 flips the model completely. Routers are not allowed to fragment. End hosts must run Path MTU Discovery before sending. The direction makes sense. But the cost shows up in practice, because PMTUD depends on ICMPv6 messages that many firewalls silently drop. That is how PMTUD black holes are born.

One detail often gets ignored in textbooks: how a receiver calculates the original payload size. The last fragment carries the answer.

original payload size = last fragment offset × 8 + last fragment data size

Say the last fragment offset is 495 and its data length is 540 bytes. The original size is 495×8+540, which gives 4500 bytes. The ×8 exists because fragment offsets are expressed in 8-byte units. Clever design, since it extends the reach of a 16-bit offset field.

Linux Kernel Fragment Reassembly Parameters, Often Left Untouched

ipfrag_high_thresh
ipfrag_low_thresh
ipfrag_time

These three control IPv4 fragment reassembly at the kernel level. ipfrag_high_thresh is the maximum memory allowed for reassembly. Once the limit is hit, the kernel starts dropping fragments until memory falls back under ipfrag_low_thresh. ipfrag_time determines how long a fragment stays in memory. The default is extremely conservative, 0xFFFFFF seconds. An attacker can exploit that to exhaust memory, which is why so many hardening guides tell you to lower it.

A common pitfall: ipfrag_low_thresh is obsolete since Linux 4.17. Plenty of old tutorials still reference it. Following them is pointless now. The kernel only cares about high_thresh threshold behavior; the old low-thresh eviction logic has been folded into newer memory management paths.

For IPv6, the equivalent names are ip6frag_high_thresh and ip6frag_low_thresh. Same mechanics, different prefix. When debugging IPv6 reassembly issues, double-check that you are reading the right sysctl nodes.

There is another parameter worth knowing: ipfrag_max_dist. It limits how much disorder is tolerated among fragments from the same source IP. A little reordering is normal. But if many fragments arrive from one source address while a fragment queue stays incomplete, that pattern usually means some fragment was lost, not merely delayed. In that case, the kernel drops the existing queue rather than waiting forever.

mipstack and Userspace Stacks, a More Aggressive Approach

If you do network development in userspace, you cannot simply copy the kernel’s strategy. MetaCubeX’s mipstack is a pure-Go userspace IP stack built for mihomo. Its fragment reassembly logic is worth studying.

Take overlapping fragments. If a new fragment overlaps an already stored range, mipstack drops the entire datagram immediately. No byte-by-byte comparison. The reasoning: legitimate traffic rarely overlaps. Overlap is usually a sign of malicious construction. Comparing payload bytes would waste time and memory, so just flush the whole thing.

Several implementation details stand out:

A duplicate final fragment can confirm the packet’s final length, but it never completes reassembly by itself.
Partial overlap or any conflict invalidates the whole reassembly state immediately.
Invalid standalone input, or a fragment belonging to another packet, leaves existing state untouched.
Incomplete fragment sets are limited by count, byte usage, piece count, and lifetime.

This engineering tradeoff contrasts sharply with the Linux kernel’s conservative stance. The kernel prefers to wait and buffer. A userspace stack prefers fast failure, because userspace memory is more precious and fragment attacks can easily exhaust a process heap.

Cisco VFR, Solving More Than MTU Problems

Cisco Catalyst SD-WAN’s Virtual Fragmentation Reassembly, or VFR, takes yet another path. Its real target is not just delivering fragments. It exists so security devices can inspect fragmented traffic.

Here is the classic security problem with fragments. Firewalls and NAT boxes need Layer 4 information to make decisions. But only the first fragment carries the Layer 4 header. All subsequent fragments are raw payload chunks. Passing them through blindly is unsafe. Dropping all of them breaks connectivity. There is no clean choice.

VFR solves this by virtually reassembling the full packet at the router, then letting security features inspect the entire payload before forwarding. Once the inspection is done, the router refragments the packet back to fit the MTU and sends it on its way. Security gets full visibility; link MTU constraints remain respected. This capability has been built into Catalyst SD-WAN since IOS XE Release 17.12.1a. In default mode, the first fragment triggers delivery of the virtually reassembled payload to each feature, and the remaining features finish the job after the final fragment arrives.

A practical takeaway for network operators: if fragmented traffic is causing problems at your firewall, do not rush to change MTUs or disable fragmentation. Check whether your security device has a virtual reassembly feature like VFR. Enabling that alone resolves a large share of such issues.

Fragment Attacks and What to Do About Them

Security. It is impossible to talk about fragmentation without addressing it. Fragmentation is a double-edged sword. Attackers abuse it to bypass security inspection or exhaust receiver resources.

Common methods include creating endless incomplete fragment sequences to consume memory, overlapping fragments to overwrite payloads already checked by firewalls, and using tiny fragments to overwhelm reassembly logic.

Linux counters such as ipfrag_high_thresh and ipfrag_time exist precisely to limit memory exhaustion. mipstack’s drop-on-overlap policy prevents overlap-based evasion attacks.

For tuning, I would suggest this order. First, identify whether any security inspection device sits on the path. Then decide whether to raise memory thresholds or fragment timeouts. If most of your traffic is TLS-encrypted anyway, fragment reassembly tuning has much less value.

Wrap-Up

IP fragmentation and reassembly span protocol design, kernel behavior, userspace implementation, and security appliances. IPv4 and IPv6 went separate ways. Linux kernel and userspace stacks have different memory strategies. Cisco VFR reassembles fragments for security inspection. The same mechanism, quite different purposes depending on where it sits.

When you hit fragmentation problems in production, do not jump straight to textbook theories. Check current kernel parameters first, then ask whether the fragments are legitimate traffic or an attack signature. Only then can you pick the right remedy.

Reference links: