General • Large receive offload breaking TCP on 6to4?

I am running a couple of 6to4 tunnels on a CCR2004-16G-2S+ and since ROS 7.16.2 (and still with 7.18) I seem to have an issue with TCP connections that is making them unusable.

The interesting thing is that it is unobservable from the router itself and we only discovered this by chance from another device.

What happens is that when TCP packets are received on the same connection at any significant enough rate on the tunnel, the payloads are combined (as by large-receive-offload) into a new packet before firewalling/routing. So, we end up with a logical packet whose size is much bigger than any MTU used. After passing through the firewall, it tries to send it through the local interface, and the resulting packet is larger than the MTU. No resegmentation occurs, the packet is dropped and an ICMP packet-too-big message is sent.

In a tcpdump done from the remote side, we can see the packet-too-big message is saying the needed MTU is 1500 (for the internal local network) but the tunnel MTU is 1480 ... and the headers of the "too-big" packet included in the ICMP packet is showing a size at least 2x normal payloads, 2700 bytes or more.

If torch or the packet sniffer is activated on the tunnel interface, the problem immediately goes away and is unobservable. The easiest way to see it is to start a download from some remote host and run tcpdump on it. The ICMP packet-too-bigs will be received multiple times per second, and the transfer rate will be terrible. On the machine I test (with 100Mbit connection) I got about a 100kB/s transfer rate, and if I activate the packet sniffer, it increases to 10MB/s

Here are the details:

• Two 6to4 tunnels to different remote IPs -- one directly to a machine I control, another to the HE tunnelbroker -- problem happens on both
• No fasttrack/fastpath used
• Internal subnets are accessed through VLANs on one of the SFP+ ports
• No bridge is used for the internal port (but I tried it with a bridge, and there was no difference)
• Connection tracking is being used
• No queues active, but using them seems to change nothing
• External interface doesn't seem to matter -- tried one ISP connection on {Ethernet->VLAN->PPPOE} and another on plain Ethernet and the behaviour is the same on both

Is anyone seeing this behaviour, too? I haven't found anything searching and it seems that this is enough of a performance killer that others would have noticed. I have thought I have been suffering from poor reputation of HE tunnelbroker addresses for bad IPv6 connectivity for ages and it turns out it is not that...

Is there any setting to turn off the large-receive-offload?

Edit to add : I just noticed that starting torch on any interface at all (including lo) also makes the problem immediately go away -- for all tunnels

Thanks

Statistics: Posted by dwplx — Sat Mar 01, 2025 3:09 pm

---