<div dir="ltr"><div class="gmail_default" style>On Fri, Jan 4, 2013 at 1:21 PM, Dave Taht <span dir="ltr"><<a href="mailto:dave.taht@gmail.com" target="_blank">dave.taht@gmail.com</a>></span> wrote:<br></div><div class="gmail_extra">
<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class="im">On Fri, Jan 4, 2013 at 12:57 PM, Jerry Chu <<a href="mailto:hkchu@google.com">hkchu@google.com</a>> wrote:<br>
> +ycheng<br>
><br>
> On Fri, Jan 4, 2013 at 12:43 PM, Maciej Soltysiak <<a href="mailto:maciej@soltysiak.com">maciej@soltysiak.com</a>><br>
> wrote:<br>
>><br>
>> Oops, apologies if email was formatted weirdly...<br>
><br>
><br>
> The problem you described below is separate from the MIPS router crash one,<br>
> right?<br>
<br>
</div>I think - but am of course unsure - that we've actually hit a<br>
different (or additional) problem than TFO - merely exposed by testing<br>
fastopen first...<br>
<br>
that said...<br>
<div class="im"><br>
>BTW, we've only tested on x86_64 arch.<br>
<br>
</div>I have multiple arches accumulated for some BQL work, mostly arm,<br>
<br>
They include a raspberry pi, a zedboard, a wndr4300, a nexus 7, and a<br>
couple other boxes in addition to my usual mips based wndr3800s and<br>
nanostation m5s. (which use mildly different mips chips and in<br>
particular the m5 does not share the unaligned rx problem that the<br>
ar71xx has a patch for). The zedboard's been problematic...<br>
<br>
So I will fold in testing tfo to looking hard at the igmp, and<br>
checksum issues seemingly exposed today. Unless someone beats me to it<br>
(I'm tied up til sunday)<br>
<br>
In the interim, at a higher layers, the current release of httpping<br>
has tfo support, and there are patches to polipo, that might benefit<br></blockquote><div><br></div><div style>Awesome!</div><div style> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
from review and wider testing on x86_64.<br>
<br>
<a href="https://raw.github.com/dtaht/ceropackages-3.3/master/net/polipo/patches/001-server_tfo.patch" target="_blank">https://raw.github.com/dtaht/ceropackages-3.3/master/net/polipo/patches/001-server_tfo.patch</a></blockquote>
<div><br></div><div>tfo_qlen is your first line of defense against spoofed TFO attack so you want to pick a value wisely. (50 seems on the safer side.)</div><div><br></div><div style>Jerry</div><div style><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
At least half of the needed support landed in netperf svn recently, too.<br>
<div class=""><div class="h5"><br>
><br>
> In addition to tcpdump, "netstat -s | grep -i fastopen" may be useful too.<br>
><br>
> Thanks,<br>
><br>
> Jerry (author of TFO server code)<br>
><br>
>><br>
>><br>
>> On Fri, Jan 4, 2013 at 9:42 PM, Maciej Soltysiak <<a href="mailto:maciej@soltysiak.com">maciej@soltysiak.com</a>><br>
>> wrote:<br>
>>><br>
>>> I am seeing something strange here, with polipo related to TFO but also<br>
>>> DNS.<br>
>>> When I just took 3.7.1-1 and set my windows 7 laptop to use<br>
>>> gw.home.lan:8123 as http proxy it didn't work. What I observed was:<br>
>>> A) after quite a while polipo's response to browser was 504 Host<br>
>>> <a href="http://www.osnews.com" target="_blank">www.osnews.com</a> lookup failed: Timeout<br>
>>> b) this error in ssh console: Host <a href="http://osnews.com" target="_blank">osnews.com</a> lookup failed: Timeout<br>
>>> (131072)<br>
>>> c) Disabling TFO by adding option useTCPFastOpen 'false' to config<br>
>>> 'polipo' 'general' works around the problem<br>
>>> d) Alternatively, you can keep TFO enabled in polipo but change option<br>
>>> 'dnsUseGethostbyname' from 'reluctantly' to 'true' (!)<br>
>>> This is very weird, because TFO is TCP and the DNS queries fired off by<br>
>>> polipo are UDP:<br>
>>> root@OpenWrt:/tmp/log# tcpdump -n -v -vv -vvv -x -X -s 1500 -i lo<br>
>>> 20:21:56.160245 IP (tos 0x0, ttl 64, id 50129, offset 0, flags [DF],<br>
>>> proto UDP (17), length 60)<br>
>>> 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd17f!] 55396+<br>
>>> A? <a href="http://www.osnews.com" target="_blank">www.osnews.com</a>. (32)<br>
>>> 0x0000: 4500 003c c3d1 4000 4011 78dd 7f00 0001 E..<..@.@.x.....<br>
>>> 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d..<br>
>>> 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn<br>
>>> 0x0030: 6577 7303 636f 6d00 0001 0001 ews.com.....<br>
>>> 20:21:56.160319 IP (tos 0x0, ttl 64, id 50130, offset 0, flags [DF],<br>
>>> proto UDP (17), length 60)<br>
>>> 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd164!] 55396+<br>
>>> AAAA? <a href="http://www.osnews.com" target="_blank">www.osnews.com</a>. (32)<br>
>>> 0x0000: 4500 003c c3d2 4000 4011 78dc 7f00 0001 E..<..@.@.x.....<br>
>>> 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d..<br>
>>> 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn<br>
>>> 0x0030: 6577 7303 636f 6d00 001c 0001 ews.com.....<br>
>>> 20:21:56.169942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto<br>
>>> UDP (17), length 123)<br>
>>> 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe7a -> 0x5f73!] 55396<br>
>>> q: A? <a href="http://www.osnews.com" target="_blank">www.osnews.com</a>. 1/2/0 <a href="http://www.osnews.com" target="_blank">www.osnews.com</a>. [29m3s] A 74.86.31.159 ns:<br>
>>> <a href="http://osnews.com" target="_blank">osnews.com</a>. [29m3s] NS <a href="http://ns2.swelter.net" target="_blank">ns2.swelter.net</a>., <a href="http://osnews.com" target="_blank">osnews.com</a>. [29m3s] NS<br>
>>> <a href="http://ns1.swelter.net" target="_blank">ns1.swelter.net</a>. (95)<br>
>>> 0x0000: 4500 007b 0000 4000 4011 3c70 7f00 0001 E..{..@.@.<p....<br>
>>> 0x0010: 7f00 0001 0035 b8c8 0067 fe7a d864 8180 .....5...g.z.d..<br>
>>> 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn<br>
>>> 0x0030: 6577 7303 636f 6d00 0001 0001 c00c 0001 ews.com.........<br>
>>> 0x0040: 0001 0000 06cf 0004 4a56 1f9f c010 0002 ........JV......<br>
>>> 0x0050: 0001 0000 06cf 0011 036e 7332 0773 7765 .........ns2.swe<br>
>>> 0x0060: 6c74 6572 036e 6574 00c0 1000 0200 0100 lter.net........<br>
>>> 0x0070: 0006 cf00 0603 6e73 31c0 40 ......ns1.@<br>
>>> 20:21:56.173901 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto<br>
>>> UDP (17), length 135)<br>
>>> 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe86 -> 0x8ecb!] 55396<br>
>>> q: AAAA? <a href="http://www.osnews.com" target="_blank">www.osnews.com</a>. 1/2/0 <a href="http://www.osnews.com" target="_blank">www.osnews.com</a>. [54m44s] AAAA<br>
>>> 2607:f0d0:1002:62::3 ns: <a href="http://osnews.com" target="_blank">osnews.com</a>. [29m3s] NS <a href="http://ns1.swelter.net" target="_blank">ns1.swelter.net</a>.,<br>
>>> <a href="http://osnews.com" target="_blank">osnews.com</a>. [29m3s] NS <a href="http://ns2.swelter.net" target="_blank">ns2.swelter.net</a>. (107)<br>
>>> 0x0000: 4500 0087 0000 4000 4011 3c64 7f00 0001 E.....@.@.<d....<br>
>>> 0x0010: 7f00 0001 0035 b8c8 0073 fe86 d864 8180 .....5...s...d..<br>
>>> 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn<br>
>>> 0x0030: 6577 7303 636f 6d00 001c 0001 c00c 001c ews.com.........<br>
>>> 0x0040: 0001 0000 0cd4 0010 2607 f0d0 1002 0062 ........&......b<br>
>>> 0x0050: 0000 0000 0000 0003 c010 0002 0001 0000 ................<br>
>>> 0x0060: 06cf 0011 036e 7331 0773 7765 6c74 6572 .....ns1.swelter<br>
>>> 0x0070: 036e 6574 00c0 1000 0200 0100 0006 cf00 .net............<br>
>>> 0x0080: 0603 6e73 32c0 4c ..ns2.L<br>
>>> This is the only DNS traffic I saw during the attempts. The tcpdumps have<br>
>>> udp bad checksum but when I disabled TFO in polipo, the UDP where still bad<br>
>>> checksum but they worked.<br>
>>> Really weird.<br>
>>> p.s. UPNP still works for port forwarding negotiation as it did in<br>
>>> 3.6.11-4<br>
>>> I still couldn't get the UPNP/SSDP broadcasts (udp to 239.255.255.250) to<br>
>>> being forwarded between se00 and sw00/sw10. Last time it worked was ~3.3.8.<br>
>>> I'm starting not to question why it doesn't work, I'm starting to wonder why<br>
>>> it did work then ;-)<br>
>>> Regards,<br>
>>> Maciej<br>
>>> On Fri, Jan 4, 2013 at 6:33 PM, Dave Taht <<a href="mailto:dave.taht@gmail.com">dave.taht@gmail.com</a>> wrote:<br>
>>>><br>
>>>> On Fri, Jan 4, 2013 at 9:27 AM, Eric Dumazet <<a href="mailto:edumazet@google.com">edumazet@google.com</a>><br>
>>>> wrote:<br>
>>>> > Sorry, could you give us a copy of the panic stack trace ?<br>
>>>><br>
>>>> I will get a serial console up on a wndr3800 by sunday. (sorry, just<br>
>>>> landed in california, am in disarray)<br>
>>>><br>
>>>> The latest dev build of cero for the wndr3800 and wndr3700v2 is at:<br>
>>>><br>
>>>> <a href="http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.7.1-1/" target="_blank">http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.7.1-1/</a><br>
>>>><br>
>>>> --<br>
>>>> Dave Täht<br>
>>>><br>
>>>> Fixing bufferbloat with cerowrt:<br>
>>>> <a href="http://www.teklibre.com/cerowrt/subscribe.html" target="_blank">http://www.teklibre.com/cerowrt/subscribe.html</a><br>
>>>> _______________________________________________<br>
>>>> Cerowrt-devel mailing list<br>
>>>> <a href="mailto:Cerowrt-devel@lists.bufferbloat.net">Cerowrt-devel@lists.bufferbloat.net</a><br>
>>>> <a href="https://lists.bufferbloat.net/listinfo/cerowrt-devel" target="_blank">https://lists.bufferbloat.net/listinfo/cerowrt-devel</a><br>
>>><br>
>>><br>
>><br>
><br>
<br>
<br>
<br>
--<br>
Dave Täht<br>
<br>
Fixing bufferbloat with cerowrt: <a href="http://www.teklibre.com/cerowrt/subscribe.html" target="_blank">http://www.teklibre.com/cerowrt/subscribe.html</a><br>
</div></div></blockquote></div><br></div></div>