Navigating MTU Concerns in IPv6 Deployments

The IPv6 header format made some radical changes away from the relative complexity of the IPv4 header. Source and destination address fields are the major change but one of the smaller changes may have a huge impact to your network. There are two flags in the IPv4 header, the “don’t fragment” bit and the “more fragments” bit, however these flags have been eliminated in the IPv6 header and this may have a huge impact on improperly configured networks. The IPv6 protocol removes the option to fragment packets that are too large and operates as if the DF-bit is always on, relying on ICMP to inform packet sources that they need to resend a packet with a smaller packet size.

Understanding MTU and Its Significance

MTU, or Maximum Transmission Unit, refers to the maximum size of a packet that can be transmitted over a network link in a single frame without needing fragmentation. Essentially, it defines the largest data payload a network can carry, including the header and trailer. MTU plays a crucial role in ensuring efficient data transmission, as smaller MTUs can lead to increased overhead due to frequent encapsulation and de-encapsulation of packets. This results in slower network performance and reduced efficiency.

MTU in IPv4 vs. IPv6

With the introduction of IPv6, there is a notable change in how MTU is handled. The IPv6 RFC mandates a minimum MTU size of 1280 bytes, this is up from a minimum MTU size of 576 bytes in the IPv4 standard. That said, the default MTU size for both protocols on ethernet interfaces is 1500 which should avoid most issues but could cause issues in the case of multiple layers of encapsulation. This increase is primarily due to the use of more extensive IPv6 headers, which are necessary for improved routing and security features. While this larger MTU has its advantages, it can also lead to compatibility issues and concerns in real-world deployments. 

Why IPv6 doesn’t fragment

I’ll be honest, I didn’t fully understand IPv6 fragmentation for a while because you’ll often read statements saying “IPv6 packets can no longer be fragmented…” when reading blogs and whitepapers on the differences between IPv4 and IPv6 but then I would get confused when reading about IPv6 fragmentation headers. So which is it? Can IPv6 be fragmented or not?

So IPv6 supports fragmentation but not it the same way as IPv4. Routers in the path of traffic can no longer break a packet in two and send along both packets when dealing with IPv6 traffic. However, if an upper layer protocol on the source client itself is trying to forward a segment that is too large for the path, the client can fragment the packet before it’s ever sent.

The reasoning behind this change was that the designers of the IPv6 protocol wanted to remove the invisible fix that is routers fragmenting packets and forwarding on the remains. They wanted to force upper layer protocols to be responsible for this path discovery rather than routers in the path of the network.

Geoff Huston of APNIC has a wonderful writeup that goes into great details of the differences of IPv4 and IPv6 fragmentation and he gives his thoughts surrounding the consequences of those protocol changes. https://blog.apnic.net/2016/01/28/evaluating-ipv4-and-ipv6-packet-frangmentation/

MTU Concerns in IPv6 Deployments

While native IPv6 ISP networks are far more common today, many ISPs still lack support. End users wishing to deploy IPv6 over carriers that haven’t deployed IPv6 may need to use a tunnel broker. Multiple layers of encapsulation have always been a problem but this is generally not a major issue in IPv4 for two reasons, first is that the IPv4 header is (usually) smaller than the IPv6 header and secondly, IPv4 allows for fragmentation whereas IPv6 does not allow for routers to fragment packets. Generally speaking, IPv4 connections with MTU issues result in poor network performance but communication still occurs, in IPv6 those same issues can result in dropped traffic.

In a network with working PMTUD (Path MTU Discovery), when a packet that is too large for a network segment, an ICMPv6 “Packet Too Big” message is returned to the packet source address telling the source to resend a smaller packet. However, because ICMP is often erroneously blocked or dropped in many networks, the traffic source will never know that its being dropped without notification and this results in a traffic black hole.

CloudFlare has a wonderful writeup on some MTU issues that they ran into with their service after enabling ECMP for load balancing anycast traffic. The issue that they were having, in a nutshell, was that traffic from their servers was hitting links with a smaller configured MTU and resulting in traffic drops. Unfortunately the ICMP PTB messages were being returned to the wrong servers due to the way that route hashing works and the server never knew to start the PMTUD process. Their solution at the time was to lower the MTU on their servers to avoid packet drops altogether until they could change their network to forward ICMP PTB to all servers to guarantee that the correct server will receive the message.

https://blog.cloudflare.com/path-mtu-discovery-in-practice

Best Practices for Mitigating MTU Concerns

1. Configure Consistent Network MTUs: Check that MTU is configured consistently across your network and large enough to accommodate your traffic needs.

2. Proper Security Configuration: Ensure that routers, firewalls, and other network devices are correctly configured to support PMTUD and handle ICMPv6 "Packet Too Big" messages. Firewalls and access-lists should not block ICMPv6 (In addition to neighbor discovery concerns). This will help prevent communication issues arising from MTU mismatches.

3. Avoid or eliminate excessive encapsulation: Multiple encapsulation protocols overlayed on top of each other will reduce the effective MTU of a link. This can especially be a problem when tunneling IPv6 inside IPv6 because of the larger header size.

4. Testing and Monitoring: Regularly test and monitor network paths for MTU-related issues using tools that simulate various packet sizes and path conditions. This can help identify potential bottlenecks or inconsistencies.

5. Adjust Servers MTU: Work with your server administrators to reduce the configured MTU. If you’re finding that there are excessive PMTUD processes being triggered and you’re unable to reduce encapsulation elsewhere in your network, this may be the solution.