[Cerowrt-devel] odhcp6c went crazy flooding Comcast with DHCPv6 SOLICITs
Dave Taht
dave.taht at gmail.com
Sat Feb 1 11:54:26 EST 2014
> On Sat, Feb 1, 2014 at 5:29 AM, Chuck Anderson <cra at wpi.edu> wrote:
>> This morning my Linux PC which has a direct connection to my Comcast
>> cable modem (no router in between) lost its IPv4 address. During
>> troubleshooting, I noticed that the dhclient was unable to get an IPv4
>> address from Comcast. I ran tcpdump and discovered that the CeroWRT
>> router, also connected to the same cable modem via a switch, was
>> flooding the WAN with DHCPv6 SOLICIT packets with about 4700
>> packets/sec, 6.6 megabits/sec of traffic! I immediately unplugged
>> CeroWRT from the WAN and then my PC was able to get an IPv4 address
>> from Comcast.
One thing to clarify the behavior you saw.
Most people are plugging in their boxes behind cerowrt exclusively
(except you and I) and running the SQM system.
One thing that happens is that all streams get
FQ'd on the way out of the router, so that (in this case) a ton of
dhcpv6 requests get deprioritized relative to the other flows.
As you were running alongside the box, rather than behind, cero
assumed it had all the bandwidth to itself and filled up your modem
to the point to where even simple dhcp traffic can't get through.
I do not doubt there is many a cerowrt/openwrt/dd-wrt/3rd party box
today flooding upstream with useless requests, with users that
don't notice anything except a slight (sub ms) delay.
You can construe this to be a feature or a bug depending
on your point of view.
>> I know CeroWrt 3.7.5-2 is old at this point, but I'm wondering if
>> something else changed to cause this behavior. Maybe Comcast
>> IPv6-enabled my CMTS finally? I've been using HE tunnels for IPv6,
>> one on a Linksys OpenWRT for my "production" network and a separate
>> tunnel on this CeroWRT for "testing".
>>
>> There is one other change that was made to my Linux PC--I booted into
>> a new kernel yesterday morning and had a similar problem with the
>> inability to get an IPv4 address via DHCP from Comcast for about 30
>> minutes, then it just started working on its own. (I hadn't noticed
>> initially since I was using IPv6 to get where I needed to go.) I
>> didn't have time to troubleshoot it at the time, but I assumed it was
>> due to this IPv6 change in the Fedora kernel:
>>
>> * Wed Jan 29 2014 Justin M. Forbes <jforbes at fedoraproject.org> - 3.12.9-201
>> - ipv6 addrconf: revert /proc/net/if_inet6 ifa_flag format (rhbz 1056711)
>>
>> * Tue Jan 28 2014 Josh Boyer <jwboyer at fedoraproject.org>
>> - Add patch from Stanislaw Gruszka to fix ath9k BUG (rhbz 990955)
>>
>> * Mon Jan 27 2014 Justin M. Forbes <jforbes at fedoraproject.org> - 3.12.9-200
>> - Backport new IPv6 address flag IFA_F_NOPREFIXROUTE and IFA_F_MANAGETEMPADDR (rhbz 1056711)
>> - Linux v3.12.9
>> - i915: remove pm_qos request on error (rhbz 1057533)
>>
>> See https://bugzilla.redhat.com/show_bug.cgi?id=1056711 for details
>> about that.
>>
>> Each time this loss of IPv4 happened, I noticed the NIC link went down
>> right before it started. Maybe the flooding was happening yesterday
>> morning too, and the flooding caused my poor 5-port Netgear switch to
>> flake out and flap the NIC links? Alternatively, maybe the link flap
>> itself was what caused odhcp6c to wig out and start flooding in the
>> first place? Unfortunately I don't have a tcpdump from yesterday
>> morning to confirm this.
>>
>> CeroWRT status:
>>
>> Router Name cerowrt
>> Router Model NETGEAR WNDR3700v2
>> Firmware Version CeroWrt Modena 3.7.5-2 / LuCI Trunk (trunk+svn)
>> Kernel Version 3.7.5
>> Local Time Sat Feb 1 07:54:43 2014
>> Uptime 58d 6h 56m 51s
>>
>> The DHCPv6 client is odhcp6c:
>>
>> root at cerowrt:~# ps |grep dhc
>> 980 root 1720 S udhcpc -p /var/run/udhcpc-ge00.pid -s /lib/netifd/dh
>> 1335 root 804 R odhcp6c -s /lib/netifd/dhcpv6.script -Ntry -P60 ge00
>> 3725 root 1704 S grep dhc
>>
>> Here is an example packet from the DHCPv6 flood:
>>
>> No. Time Source Destination Protocol Length Info
>> 1 0.000000 fe80::c63d:c7ff:feb0:8f41 ff02::1:2 DHCPv6 179 Solicit XID: 0x45eb91 CID: 00030001c43dc7b08f41
>>
>> Frame 1: 179 bytes on wire (1432 bits), 179 bytes captured (1432 bits)
>> Encapsulation type: Ethernet (1)
>> Arrival Time: Feb 1, 2014 07:20:27.723633000 EST
>> [Time shift for this packet: 0.000000000 seconds]
>> Epoch Time: 1391257227.723633000 seconds
>> [Time delta from previous captured frame: 0.000000000 seconds]
>> [Time delta from previous displayed frame: 0.000000000 seconds]
>> [Time since reference or first frame: 0.000000000 seconds]
>> Frame Number: 1
>> Frame Length: 179 bytes (1432 bits)
>> Capture Length: 179 bytes (1432 bits)
>> [Frame is marked: False]
>> [Frame is ignored: False]
>> [Protocols in frame: eth:ipv6:udp:dhcpv6]
>> [Coloring Rule Name: UDP]
>> [Coloring Rule String: udp]
>> Ethernet II, Src: Netgear_b0:8f:41 (c4:3d:c7:b0:8f:41), Dst: IPv6mcast_00:01:00:02 (33:33:00:01:00:02)
>> Destination: IPv6mcast_00:01:00:02 (33:33:00:01:00:02)
>> Address: IPv6mcast_00:01:00:02 (33:33:00:01:00:02)
>> .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
>> .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
>> Source: Netgear_b0:8f:41 (c4:3d:c7:b0:8f:41)
>> Address: Netgear_b0:8f:41 (c4:3d:c7:b0:8f:41)
>> .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
>> Type: IPv6 (0x86dd)
>> Internet Protocol Version 6, Src: fe80::c63d:c7ff:feb0:8f41 (fe80::c63d:c7ff:feb0:8f41), Dst: ff02::1:2 (ff02::1:2)
>> 0110 .... = Version: 6
>> [0110 .... = This field makes the filter "ip.version == 6" possible: 6]
>> .... 0000 0000 .... .... .... .... .... = Traffic class: 0x00000000
>> .... 0000 00.. .... .... .... .... .... = Differentiated Services Field: Default (0x00000000)
>> .... .... ..0. .... .... .... .... .... = ECN-Capable Transport (ECT): Not set
>> .... .... ...0 .... .... .... .... .... = ECN-CE: Not set
>> .... .... .... 0000 0000 0000 0000 0000 = Flowlabel: 0x00000000
>> Payload length: 125
>> Next header: UDP (17)
>> Hop limit: 1
>> Source: fe80::c63d:c7ff:feb0:8f41 (fe80::c63d:c7ff:feb0:8f41)
>> [Source SA MAC: Netgear_b0:8f:41 (c4:3d:c7:b0:8f:41)]
>> Destination: ff02::1:2 (ff02::1:2)
>> [Source GeoIP: Unknown]
>> [Destination GeoIP: Unknown]
>> User Datagram Protocol, Src Port: dhcpv6-client (546), Dst Port: dhcpv6-server (547)
>> Source port: dhcpv6-client (546)
>> Destination port: dhcpv6-server (547)
>> Length: 125
>> Checksum: 0xda1c [validation disabled]
>> [Good Checksum: False]
>> [Bad Checksum: False]
>> DHCPv6
>> Message type: Solicit (1)
>> Transaction ID: 0x45eb91
>> Elapsed time
>> Option: Elapsed time (8)
>> Length: 2
>> Value: ffff
>> Elapsed-time: 655350 ms
>> Option Request
>> Option: Option Request (6)
>> Length: 10
>> Value: 00170018003800160015
>> Requested Option code: DNS recursive name server (23)
>> Requested Option code: Domain Search List (24)
>> Requested Option code: Unknown (56)
>> Requested Option code: SIP Servers IPv6 Address List (22)
>> Requested Option code: SIP Server Domain Name List (21)
>> Client Identifier
>> Option: Client Identifier (1)
>> Length: 10
>> Value: 00030001c43dc7b08f41
>> DUID: 00030001c43dc7b08f41
>> DUID Type: link-layer address (3)
>> Hardware type: Ethernet (1)
>> Link-layer address: c4:3d:c7:b0:8f:41
>> Reconfigure Accept
>> Option: Reconfigure Accept (20)
>> Length: 0
>> Fully Qualified Domain Name
>> Option: Fully Qualified Domain Name (39)
>> Length: 10
>> Value: 00076365726f77727400
>> 0000 0... = Reserved: 0x00
>> .... .0.. = N bit: Server should perform DNS updates
>> .... ..0. = O bit: Server has not overridden client's S bit preference
>> .... ...0 = S bit: Server should not perform forward DNS updates
>> Domain: cerowrt
>> Identity Association for Non-temporary Address
>> Option: Identity Association for Non-temporary Address (3)
>> Length: 12
>> Value: 000000010000000000000000
>> IAID: 00000001
>> T1: 0
>> T2: 0
>> Identity Association for Prefix Delegation
>> Option: Identity Association for Prefix Delegation (25)
>> Length: 41
>> Value: 000000010000000000000000001a00190000000000000000...
>> IAID: 00000001
>> T1: 0
>> T2: 0
>> IA Prefix
>> Option: IA Prefix (26)
>> Length: 25
>> Value: 00000000000000003c000000000000000000000000000000...
>> Preferred lifetime: 0
>> Valid lifetime: 0
>> Prefix length: 60
>> Prefix address: :: (::)
>>
>> 0000 33 33 00 01 00 02 c4 3d c7 b0 8f 41 86 dd 60 00 33.....=...A..`.
>> 0010 00 00 00 7d 11 01 fe 80 00 00 00 00 00 00 c6 3d ...}...........=
>> 0020 c7 ff fe b0 8f 41 ff 02 00 00 00 00 00 00 00 00 .....A..........
>> 0030 00 00 00 01 00 02 02 22 02 23 00 7d da 1c 01 45 .......".#.}...E
>> 0040 eb 91 00 08 00 02 ff ff 00 06 00 0a 00 17 00 18 ................
>> 0050 00 38 00 16 00 15 00 01 00 0a 00 03 00 01 c4 3d .8.............=
>> 0060 c7 b0 8f 41 00 14 00 00 00 27 00 0a 00 07 63 65 ...A.....'....ce
>> 0070 72 6f 77 72 74 00 00 03 00 0c 00 00 00 01 00 00 rowrt...........
>> 0080 00 00 00 00 00 00 00 19 00 29 00 00 00 01 00 00 .........)......
>> 0090 00 00 00 00 00 00 00 1a 00 19 00 00 00 00 00 00 ................
>> 00a0 00 00 3c 00 00 00 00 00 00 00 00 00 00 00 00 00 ..<.............
>> 00b0 00 00 00 ...
>>
>> The CeroWRT system log is attached. Nothing looks strange except the
>> loss of ge00 link around 6:24 this morning, which is right around when
>> I lost IPv4 connectivity to my Linux PC (I have a system monitoring
>> this IP and it SMS's me if it goes down). My PC's NIC link went down
>> at exactly the same time. At 7:24 is when I unplugged CeroWRT.
>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
--
Dave Täht
Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
More information about the Cerowrt-devel
mailing list