From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f42.google.com (mail-la0-f42.google.com [209.85.215.42]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id A8B7621F0F2 for ; Fri, 4 Jan 2013 12:57:26 -0800 (PST) Received: by mail-la0-f42.google.com with SMTP id fe20so10429163lab.29 for ; Fri, 04 Jan 2013 12:57:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DKhiW50uJEwK+dp7zHPJy7E/26ucTLjX2EY6HhqZcBk=; b=k3EtHxZU5GpPlE6PuuWbbT7acCotLaJEE+F5SapI6EL51pG+JhcRGNokgCvZ2tl/SC WKVFQ4jskZf4oIGUNI8/Q1dkHT1bhYTmS12nkMQeBdBXcAosli1VuFp9Hs9yAFxjUr+p VCufRUT+HZSt2Jk5QG6oACeU2hhmhYiBR13Hy55e5B+VUgwLyJCgtRjs8ujhLJGXEtA6 nvzive7C15FDpjJst7PEkbgtKR1qhyJeculgFYIM7i8EbAfMgm0ohYMRFFJHHORPPJQz 13eJ9ttmq3rnyckHisvzVZHsvD9tobDusxlC6+WUUto1U8JOW25WD3ZLwm6wJ7pyVGgk NGCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=DKhiW50uJEwK+dp7zHPJy7E/26ucTLjX2EY6HhqZcBk=; b=OdWnHmZDua/9LnO3NZCkkeNq+QdeRdJ5FyMA70zs9zRwWo0wbHU7J0qpZ+4N6ofcrd GwG/vj2kaTBGVgAIk3kLZQeyj7sp3xBpzb6xcD9JXFqPOmPzGuq+Du+38K0N9yTxJ64J glw9L+X+oo3huhazdzONlCbVBwNwtrhW/i+40zvxw1belp5khMoSH5Sbmgoi9xEelrPP 6zH/vamwuc1XRbrVP4YwrJRoHGwSczEYWlQLJfDmLIlMtWZKx3eC/f7GFoLmhuRGbrXD FKCQlbBWU8JfqTJKoB7jTZocWT6H4MyjrzMW6oarvY411m1W5ECpnQxDQR1ZvZJWxXSE MAwg== MIME-Version: 1.0 Received: by 10.152.124.68 with SMTP id mg4mr50245204lab.51.1357333044278; Fri, 04 Jan 2013 12:57:24 -0800 (PST) Received: by 10.112.49.134 with HTTP; Fri, 4 Jan 2013 12:57:23 -0800 (PST) In-Reply-To: References: Date: Fri, 4 Jan 2013 12:57:23 -0800 Message-ID: From: Jerry Chu To: Maciej Soltysiak Content-Type: multipart/alternative; boundary=f46d043bd8fef2374604d27cb93b X-Gm-Message-State: ALoCoQnygvwofjhCFL+uNPNJlb25OgTFgC725F54KusVyFz9vu/2nOpw6Z8OTUooJ7N9p4J+cW+uTLh5VDkHzU/vdwoS0U8CfZJFf5k3kkrwas27omyDBIQkc0CbQhkK/u+lBhIBKKlby1FF98kRNwY3W4BaLqkRhvu1HAELRyHDhg4Oe7Lf6iPUvfUSVU+xQ42eq4vP+DF3MfATNdzfVtHv10buEPDYLg== X-Mailman-Approved-At: Fri, 04 Jan 2013 13:45:31 -0800 Cc: Yuchung Cheng , Eric Dumazet , cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] TFO crashes cerowrt 3.7.1-1 X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2013 20:57:27 -0000 --f46d043bd8fef2374604d27cb93b Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable +ycheng On Fri, Jan 4, 2013 at 12:43 PM, Maciej Soltysiak wro= te: > Oops, apologies if email was formatted weirdly... The problem you described below is separate from the MIPS router crash one, right? BTW, we've only tested on x86_64 arch. In addition to tcpdump, "netstat -s | grep -i fastopen" may be useful too. Thanks, Jerry (author of TFO server code) > > On Fri, Jan 4, 2013 at 9:42 PM, Maciej Soltysiak wr= ote: > >> I am seeing something strange here, with polipo related to TFO but also >> DNS. >> When I just took 3.7.1-1 and set my windows 7 laptop to use >> gw.home.lan:8123 as http proxy it didn't work. What I observed was: >> A) after quite a while polipo's response to browser was 504 Host >> www.osnews.com lookup failed: Timeout >> b) this error in ssh console: Host osnews.com lookup failed: Timeout >> (131072) >> c) Disabling TFO by adding option useTCPFastOpen 'false' to config >> 'polipo' 'general' works around the problem >> d) Alternatively, you can keep TFO enabled in polipo but change option >> 'dnsUseGethostbyname' from 'reluctantly' to 'true' (!) >> This is very weird, because TFO is TCP and the DNS queries fired off by >> polipo are UDP: >> root@OpenWrt:/tmp/log# tcpdump -n -v -vv -vvv -x -X -s 1500 -i lo >> 20:21:56.160245 IP (tos 0x0, ttl 64, id 50129, offset 0, flags [DF], >> proto UDP (17), length 60) >> 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd17f!] 55396+ >> A? www.osnews.com. (32) >> 0x0000: 4500 003c c3d1 4000 4011 78dd 7f00 0001 E..<..@.@.x..... >> 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d.. >> 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn >> 0x0030: 6577 7303 636f 6d00 0001 0001 ews.com..... >> 20:21:56.160319 IP (tos 0x0, ttl 64, id 50130, offset 0, flags [DF], >> proto UDP (17), length 60) >> 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd164!] 55396+ >> AAAA? www.osnews.com. (32) >> 0x0000: 4500 003c c3d2 4000 4011 78dc 7f00 0001 E..<..@.@.x..... >> 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d.. >> 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn >> 0x0030: 6577 7303 636f 6d00 001c 0001 ews.com..... >> 20:21:56.169942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto >> UDP (17), length 123) >> 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe7a -> 0x5f73!] 55396 >> q: A? www.osnews.com. 1/2/0 www.osnews.com. [29m3s] A 74.86.31.159 ns: >> osnews.com. [29m3s] NS ns2.swelter.net., osnews.com. [29m3s] NS >> ns1.swelter.net. (95) >> 0x0000: 4500 007b 0000 4000 4011 3c70 7f00 0001 E..{..@.@.> 0x0010: 7f00 0001 0035 b8c8 0067 fe7a d864 8180 .....5...g.z.d.. >> 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn >> 0x0030: 6577 7303 636f 6d00 0001 0001 c00c 0001 ews.com......... >> 0x0040: 0001 0000 06cf 0004 4a56 1f9f c010 0002 ........JV...... >> 0x0050: 0001 0000 06cf 0011 036e 7332 0773 7765 .........ns2.swe >> 0x0060: 6c74 6572 036e 6574 00c0 1000 0200 0100 lter.net........ >> 0x0070: 0006 cf00 0603 6e73 31c0 40 ......ns1.@ >> 20:21:56.173901 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto >> UDP (17), length 135) >> 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe86 -> 0x8ecb!] 55396 >> q: AAAA? www.osnews.com. 1/2/0 www.osnews.com. [54m44s] AAAA >> 2607:f0d0:1002:62::3 ns: osnews.com. [29m3s] NS ns1.swelter.net., >> osnews.com. [29m3s] NS ns2.swelter.net. (107) >> 0x0000: 4500 0087 0000 4000 4011 3c64 7f00 0001 E.....@.@.> 0x0010: 7f00 0001 0035 b8c8 0073 fe86 d864 8180 .....5...s...d.. >> 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn >> 0x0030: 6577 7303 636f 6d00 001c 0001 c00c 001c ews.com......... >> 0x0040: 0001 0000 0cd4 0010 2607 f0d0 1002 0062 ........&......b >> 0x0050: 0000 0000 0000 0003 c010 0002 0001 0000 ................ >> 0x0060: 06cf 0011 036e 7331 0773 7765 6c74 6572 .....ns1.swelter >> 0x0070: 036e 6574 00c0 1000 0200 0100 0006 cf00 .net............ >> 0x0080: 0603 6e73 32c0 4c ..ns2.L >> This is the only DNS traffic I saw during the attempts. The tcpdumps hav= e >> udp bad checksum but when I disabled TFO in polipo, the UDP where still = bad >> checksum but they worked. >> Really weird. >> p.s. UPNP still works for port forwarding negotiation as it did in >> 3.6.11-4 >> I still couldn't get the UPNP/SSDP broadcasts (udp to 239.255.255.250) t= o >> being forwarded between se00 and sw00/sw10. Last time it worked was ~3.3= .8. >> I'm starting not to question why it doesn't work, I'm starting to wonder >> why it did work then ;-) >> Regards, >> Maciej >> On Fri, Jan 4, 2013 at 6:33 PM, Dave Taht wrote: >> >>> On Fri, Jan 4, 2013 at 9:27 AM, Eric Dumazet >>> wrote: >>> > Sorry, could you give us a copy of the panic stack trace ? >>> >>> I will get a serial console up on a wndr3800 by sunday. (sorry, just >>> landed in california, am in disarray) >>> >>> The latest dev build of cero for the wndr3800 and wndr3700v2 is at: >>> >>> http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.7.1-1/ >>> >>> -- >>> Dave T=E4ht >>> >>> Fixing bufferbloat with cerowrt: >>> http://www.teklibre.com/cerowrt/subscribe.html >>> _______________________________________________ >>> Cerowrt-devel mailing list >>> Cerowrt-devel@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>> >> >> > --f46d043bd8fef2374604d27cb93b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
+ycheng

On Fri, Jan 4, 2013 at 12:4= 3 PM, Maciej Soltysiak <maciej@soltysiak.com> wrote:
Oops, apologies if email was formatted weird= ly...

The problem you described below= is separate from the MIPS router crash one, right? BTW, we've only tes= ted on x86_64 arch.

In addition to tcpdump, "netstat -s | = grep -i fastopen" may be useful too.

Thanks,

Jerry (author of TFO serve= r code)

<= div class=3D"h5">

On Fri, Jan 4, 2013 at = 9:42 PM, Maciej Soltysiak <maciej@soltysiak.com> wrote:
I am seeing something strange here, with polipo relat= ed to TFO but also DNS.
When I just took 3.7.1-1 and set my windows 7 laptop to us= e gw.home.lan:8123 as http proxy it didn't work. What I observed was:
A) after quite a while polipo's response to browser was 504 = Host www.osnews.com= lookup failed: Timeout
b) this error in ssh console: Host= osnews.com lookup fail= ed: Timeout (131072)
c) Disabling TFO by adding option useTCPFastOpen 'false' to co= nfig 'polipo' 'general' works around the problem
= d) Alternatively, you can keep TFO enabled in polipo but change option '= ;dnsUseGethostbyname' from 'reluctantly' to 'true' (!)<= /div>
This is very weird, because TFO is TCP and the DNS queries= fired off by polipo are UDP:
root@OpenWrt:/tmp/log# tcpdump -n = -v -vv -vvv -x -X -s 1500 -i lo
20:21:56.160245 IP (tos 0x0, ttl 64, id 50129, offset 0, flags [DF], p= roto UDP (17), length 60)
127.0.0.1.47304 > 127.0.0.1.53: [bad ud= p cksum 0xfe3b -> 0xd17f!] 55396+ A? www.osnews.com. (32)
0x0000: 4500 003c c3d1 4000 4011 78dd 7f00 0001 E..<..@.@.x.....
0x0010: 7f0= 0 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d..
0x0020: = 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn
0x0030: 6577 7303 636f 6d00 0001 0001 ews.com.....
2= 0:21:56.160319 IP (tos 0x0, ttl 64, id 50130, offset 0, flags [DF], proto U= DP (17), length 60)
127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksu= m 0xfe3b -> 0xd164!] 55396+ AAAA? www.osnews.com. (32)
0x0000: 4500 003c c3d2 4000 4011 78dc 7f00 0001 E..<..@.@.x.....
0x0010: 7f0= 0 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d..
0x0020: = 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn
0x0030: 6577 7303 636f 6d00 001c 0001 ews.com.....
2= 0:21:56.169942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (= 17), length 123)
127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0= xfe7a -> 0x5f73!] 55396 q: A? www.osnews.com. 1/2/0 www.osnews.com. [29m3s] A 74.86.31.159 ns: osnews.com. [29m3s] NS ns2.swelter.net., osnews.com. [29m3s] NS ns1.swelter.net. (95)
0x0000: 4500 007b 0000 4000 4011 3c70 7f00 0001 E..{..@.@.<p....
0x0= 010: 7f00 0001 0035 b8c8 0067 fe7a d864 8180 .....5...g.z.d..
= 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn
0x0030: 6577 7303 636f 6d00 0001 0001 c00c 0001 ews.com.........<= br> 0x0040: 0001 0000 06cf 0004 4a56 1f9f c010 0002 ........JV....= ..
0x0050: 0001 0000 06cf 0011 036e 7332 0773 7765 .........ns= 2.swe
0x0060: 6c74 6572 036e 6574 00c0 1000 0200 0100 lter.net........<= br> 0x0070: 0006 cf00 0603 6e73 31c0 40 ......ns1.@20:21:56.173901 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP= (17), length 135)
127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe86 -> 0x8ecb!]= 55396 q: AAAA? www.os= news.com. 1/2/0 ww= w.osnews.com. [54m44s] AAAA 2607:f0d0:1002:62::3 ns: osnews.com. [29m3s] NS ns1.swelter.net., osnews.com. [29m3s] NS ns2.swelter.net. (107)
0x0000: 4500 0087 0000 4000 4011 3c64 7f00 0001 E.....@.@.<d....
0x001= 0: 7f00 0001 0035 b8c8 0073 fe86 d864 8180 .....5...s...d..
0x= 0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn
0x0030: 6577 7303 636f 6d00 001c 0001 c00c 001c ews.com.........<= br> 0x0040: 0001 0000 0cd4 0010 2607 f0d0 1002 0062 ........&.= .....b
0x0050: 0000 0000 0000 0003 c010 0002 0001 0000 .......= .........
0x0060: 06cf 0011 036e 7331 0773 7765 6c74 6572 .....ns1.swelter<= br> 0x0070: 036e 6574 00c0 1000 0200 0100 0006 cf00 .net..........= ..
0x0080: 0603 6e73 32c0 4c ..ns2.L
This is the only DNS traffic I saw during the attempts. The tcpd= umps have udp bad checksum but when I disabled TFO in polipo, the UDP where= still bad checksum but they worked.
Really weird.
p.s. UPNP still works for port forwarding negotiation as it did = in 3.6.11-4
I still couldn't get the UPNP/SSDP broadcasts (ud= p to 239.255.255.250) to being forwarded between se00 and sw00/sw10. Last t= ime it worked was ~3.3.8. I'm starting not to question why it doesn'= ;t work, I'm starting to wonder why it did work then ;-)
Regards,
Maciej


--f46d043bd8fef2374604d27cb93b--