From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp191.iad.emailsrvr.com (smtp191.iad.emailsrvr.com [207.97.245.191]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id E178321F0F2 for ; Fri, 4 Jan 2013 13:01:02 -0800 (PST) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp39.relay.iad1a.emailsrvr.com (SMTP Server) with ESMTP id 9767F981FF; Fri, 4 Jan 2013 16:01:01 -0500 (EST) X-Virus-Scanned: OK Received: from legacy5.wa-web.iad1a (legacy5.wa-web.iad1a.rsapps.net [192.168.2.221]) by smtp39.relay.iad1a.emailsrvr.com (SMTP Server) with ESMTP id 5F054981B2; Fri, 4 Jan 2013 16:01:01 -0500 (EST) Received: from reed.com (localhost [127.0.0.1]) by legacy5.wa-web.iad1a (Postfix) with ESMTP id 2EC842E9802E; Fri, 4 Jan 2013 16:01:01 -0500 (EST) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Fri, 4 Jan 2013 16:01:01 -0500 (EST) Date: Fri, 4 Jan 2013 16:01:01 -0500 (EST) From: dpreed@reed.com To: "Maciej Soltysiak" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20130104160101000000_54591" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: Message-ID: <1357333261.187724652@apps.rackspace.com> X-Mailer: webmail7.0 Cc: Jerry Chu , Eric Dumazet , cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] TFO crashes cerowrt 3.7.1-1 X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2013 21:01:03 -0000 ------=_20130104160101000000_54591 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AIs this a TFO where the endpoint is on cerowrt, or just a SYN+DATA for a= non cerowrt destination?=0A =0AI was looking at the firewall rules, and th= ey are pretty complicated. Perhaps the SYN+DATA triggers a strange firewal= l behavior (a loop?) SYN's are special to firewalls, as we know.=0A =0A--= ---Original Message-----=0AFrom: "Maciej Soltysiak" = =0ASent: Friday, January 4, 2013 3:43pm=0ATo: "Dave Taht" , "Ketan Kulkarni" =0ACc: "Jerry Chu" , "Eric Dumazet" , cerowrt-devel@lists.bufferbloa= t.net=0ASubject: Re: [Cerowrt-devel] TFO crashes cerowrt 3.7.1-1=0A=0A=0A= =0AOops, apologies if email was formatted weirdly...=0A=0A=0AOn Fri, Jan 4,= 2013 at 9:42 PM, Maciej Soltysiak <[mailto:maciej@soltysiak.com] maciej@so= ltysiak.com> wrote:=0A=0AI am seeing something strange here, with polipo re= lated to TFO but also DNS.=0A=0AWhen I just took 3.7.1-1 and set my windows= 7 laptop to use gw.home.lan:8123 as http proxy it didn't work. What I obse= rved was:=0AA) after quite a while polipo's response to browser was 504 Hos= t [http://www.osnews.com] www.osnews.com lookup failed: Timeout=0Ab) this e= rror in ssh console: Host [http://osnews.com] osnews.com lookup failed: Tim= eout (131072)=0Ac) Disabling TFO by adding option useTCPFastOpen 'false' to= config 'polipo' 'general' works around the problem=0Ad) Alternatively, you= can keep TFO enabled in polipo but change option 'dnsUseGethostbyname' fro= m 'reluctantly' to 'true' (!)=0A=0AThis is very weird, because TFO is TCP a= nd the DNS queries fired off by polipo are UDP:=0A=0A[mailto:root@OpenWrt:/= tmp/log#] root@OpenWrt:/tmp/log# tcpdump -n -v -vv -vvv -x -X -s 1500 -i lo= =0A20:21:56.160245 IP (tos 0x0, ttl 64, id 50129, offset 0, flags [DF], pro= to UDP (17), length 60)=0A 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0= xfe3b -> 0xd17f!] 55396+ A? [http://www.osnews.com/] www.osnews.com. (32)= =0A 0x0000: 4500 003c c3d1 4000 4011 78dd 7f00 0001 E..<[mailto:..@.@.x] = ..@.@.x.....=0A 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.= (.;.d..=0A 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.o= sn=0A 0x0030: 6577 7303 636f 6d00 0001 0001 ews.com.....=0A20:2= 1:56.160319 IP (tos 0x0, ttl 64, id 50130, offset 0, flags [DF], proto UDP = (17), length 60)=0A 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -= > 0xd164!] 55396+ AAAA? [http://www.osnews.com/] www.osnews.com. (32)=0A 0x= 0000: 4500 003c c3d2 4000 4011 78dc 7f00 0001 E..<[mailto:..@.@.x] ..@.@.= x.....=0A 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d.= .=0A 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn=0A = 0x0030: 6577 7303 636f 6d00 001c 0001 ews.com.....=0A20:21:56.1= 69942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), leng= th 123)=0A 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe7a -> 0x5f73!= ] 55396 q: A? [http://www.osnews.com/] www.osnews.com. 1/2/0 [http://www.os= news.com/] www.osnews.com. [29m3s] A 74.86.31.159 ns: [http://osnews.com] o= snews.com. [29m3s] NS [http://ns2.swelter.net] ns2.swelter.net., [http://os= news.com] osnews.com. [29m3s] NS [http://ns1.swelter.net] ns1.swelter.net. = (95)=0A 0x0000: 4500 007b 0000 4000 4011 3c70 7f00 0001 [mailto:E..%7B..@= .@.%3Cp] E..{..@.@. 127.0.0.1.47304: [bad udp cksum 0xfe86 -> 0x8ecb!] = 55396 q: AAAA? [http://www.osnews.com/] www.osnews.com. 1/2/0 [http://www.o= snews.com/] www.osnews.com. [54m44s] AAAA 2607:f0d0:1002:62::3 ns: [http://= osnews.com] osnews.com. [29m3s] NS [http://ns1.swelter.net] ns1.swelter.net= ., [http://osnews.com] osnews.com. [29m3s] NS [http://ns2.swelter.net] ns2.= swelter.net. (107)=0A 0x0000: 4500 0087 0000 4000 4011 3c64 7f00 0001 [ma= ilto:E.....@.@.%3Cd] E.....@.@. wrote:=0A=0AOn Fri, Jan 4, 2013 at= 9:27 AM, Eric Dumazet <[mailto:edumazet@google.com] edumazet@google.com> w= rote:=0A > Sorry, could you give us a copy of the panic stack trace ?=0A=0A= I will get a serial console up on a wndr3800 by sunday. (sorry, just=0Aland= ed in california, am in disarray)=0A=0AThe latest dev build of cero for the= wndr3800 and wndr3700v2 is at:=0A=0A[http://snapon.lab.bufferbloat.net/~ce= ro2/cerowrt/wndr/3.7.1-1/] http://snapon.lab.bufferbloat.net/~cero2/cerowrt= /wndr/3.7.1-1/=0A=0A=0A=0A--=0ADave T=C3=A4ht=0A=0AFixing bufferbloat with = cerowrt: [http://www.teklibre.com/cerowrt/subscribe.html] http://www.teklib= re.com/cerowrt/subscribe.html=0A __________________________________________= _____=0ACerowrt-devel mailing list=0A[mailto:Cerowrt-devel@lists.bufferbloa= t.net] Cerowrt-devel@lists.bufferbloat.net=0A[https://lists.bufferbloat.net= /listinfo/cerowrt-devel] https://lists.bufferbloat.net/listinfo/cerowrt-dev= el ------=_20130104160101000000_54591 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

= Is this a TFO where the endpoint is on cerowrt, or just a SYN+DATA for a no= n cerowrt destination?

=0A

 

=0A=

I was looking at the firewall rules, and t= hey are pretty complicated.  Perhaps the SYN+DATA triggers a strange f= irewall behavior (a loop?)   SYN's are special to firewalls, as w= e know.

=0A

 

=0A

-----Original Message-----
From: "Maciej Soltysiak" = <maciej@soltysiak.com>
Sent: Friday, January 4, 2013 3:43pm
To: "Dave Taht" <dave.taht@gmail.com>, "Ketan Kulkarni" <ketkulka= @gmail.com>
Cc: "Jerry Chu" <hkchu@google.com>, "Eric Dumazet= " <edumazet@google.com>, cerowrt-devel@lists.bufferbloat.net
Sub= ject: Re: [Cerowrt-devel] TFO crashes cerowrt 3.7.1-1

=0A=0A

Oops, apo= logies if email was formatted weirdly...

=0A
On Fri, Jan 4, 2013 at 9:42 PM, Maciej Soltysiak <maciej@sol= tysiak.com> wrote:
=0A
=0A
I am see= ing something strange here, with polipo related to TFO but also DNS.
= =0A
=0A
When I just took 3.7.1-1 and set my windows 7 laptop = to use gw.home.lan:8123 as http proxy it didn't work. What I observed was:<= /div>=0A
A) after quite a while polipo's response to browser was 504 Host www.osnews.com= lookup failed: Timeout
=0A
b) this error in ssh consol= e: Host osnews.com look= up failed: Timeout (131072)
=0A
c) Disabling TFO by adding option = useTCPFastOpen 'false' to config 'polipo' 'general' works around the proble= m
=0A
d) Alternatively, you can keep TFO enabled in polipo but cha= nge option 'dnsUseGethostbyname' from 'reluctantly' to 'true' (!)
=0A<= div>
=0A
This is very weird, because TFO is TCP and the DNS querie= s fired off by polipo are UDP:
=0A
=0A
root@OpenWrt:/tmp/log# tcpdu= mp -n -v -vv -vvv -x -X -s 1500 -i lo
=0A
20:21:56.160245 IP (tos = 0x0, ttl 64, id 50129, offset 0, flags [DF], proto UDP (17), length 60)
127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd17f!] = 55396+ A? www.osnews.c= om. (32)
0x0000: 4500 003c c3d1 4000 4011 78dd 7f00 0001 E..<= ;..@.@.x.....
0x0010= : 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d..
0x0020: = 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn
0x0030: 6= 577 7303 636f 6d00 0001 0001 ews.com.....
20:21:56.160319 I= P (tos 0x0, ttl 64, id 50130, offset 0, flags [DF], proto UDP (17), length = 60)
127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0x= d164!] 55396+ AAAA? ww= w.osnews.com. (32)
0x0000: 4500 003c c3d2 4000 4011 78dc 7f00 00= 01 E..<..@.@.x..... 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d..
0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn
= 0x0030: 6577 7303 636f 6d00 001c 0001 ews.com.....
20:21:5= 6.169942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), l= ength 123)
127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe7a -= > 0x5f73!] 55396 q: A? www.osnews.com. 1/2/0 www.osnews.com. [29m3s] A 74.86.31.159 ns: osnews.com. [29m3s] NS ns2.swelter.net., osnews.com. [29m3s] NS ns1.swelter.net. (95)
0x0000: 4= 500 007b 0000 4000 4011 3c70 7f00 0001 E..{..@.@.<p....
0x0010: 7f00 0001 0035 b8= c8 0067 fe7a d864 8180 .....5...g.z.d..
0x0020: 0001 0001 0002 0000= 0377 7777 066f 736e .........www.osn
0x0030: 6577 7303 636f 6d00 0= 001 0001 c00c 0001 ews.com.........
0x0040: 0001 0000 06cf 0004 4a5= 6 1f9f c010 0002 ........JV......
0x0050: 0001 0000 06cf 0011 036e = 7332 0773 7765 .........ns2.swe
0x0060: 6c74 6572 036e 6574 00c0 10= 00 0200 0100 lter.net........
0x0070: 0006 cf00 0603 6e73 31c0 40 = ......ns1.@
20:21:56.173901 IP (tos 0x0, ttl 64, id 0, off= set 0, flags [DF], proto UDP (17), length 135)
127.0.0.1.53 > 127.= 0.0.1.47304: [bad udp cksum 0xfe86 -> 0x8ecb!] 55396 q: AAAA? www.osnews.com. 1/2/0 www.osnews.com. [54m44s]= AAAA 2607:f0d0:1002:62::3 ns: osnews.com. [29m3s] NS ns1.swelter.net., osnews.com. [29m3s] NS ns2.swelter.net. (107)
0x0000: 4500 0087 0000 4000 4011= 3c64 7f00 0001 E.....= @.@.<d....
0x0010: 7f00 0001 0035 b8c8 0073 fe86 d864 8180 .= ....5...s...d..
0x0020: 0001 0001 0002 0000 0377 7777 066f 736e ...= ......www.osn
0x0030: 6577 7303 636f 6d00 001c 0001 c00c 001c ews.c= om.........
0x0040: 0001 0000 0cd4 0010 2607 f0d0 1002 0062 .......= .&......b
0x0050: 0000 0000 0000 0003 c010 0002 0001 0000 .....= ...........
0x0060: 06cf 0011 036e 7331 0773 7765 6c74 6572 .....ns= 1.swelter
0x0070: 036e 6574 00c0 1000 0200 0100 0006 cf00 .net.....= .......
0x0080: 0603 6e73 32c0 4c ..ns2.L=0A
This is the only DNS traffic I saw during the attempts. The tcpdu= mps have udp bad checksum but when I disabled TFO in polipo, the UDP where = still bad checksum but they worked.
=0A
=0A
Really weird= .
=0A
p.s. UPNP still works for port forwarding negotiation as it = did in 3.6.11-4
=0A
I still couldn't get the UPNP/SSDP broadcasts = (udp to 239.255.255.250) to being forwarded between se00 and sw00/sw10. Las= t time it worked was ~3.3.8. I'm starting not to question why it doesn't wo= rk, I'm starting to wonder why it did work then ;-)
=0A
=0A<= div>Regards,
=0A
Maciej
=0A
=0A
=0A
=0A
On Fri, Jan 4, 2013 at= 6:33 PM, Dave Taht <dave.taht@gmail.com> wrote:
=0A<= blockquote class=3D"gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; paddin= g-left: 1ex; border-left-color: #cccccc; border-left-width: 1px; border-lef= t-style: solid;">=0A
On Fri, Jan 4, 2013 at 9:27 AM, Eric Dumazet <<= a href=3D"mailto:edumazet@google.com" target=3D"_blank">edumazet@google.com= > wrote:
> Sorry, could you give us a copy of the panic sta= ck trace ?

=0AI will get a serial console up on a wndr3800= by sunday. (sorry, just
landed in california, am in disarray)
The latest dev build of cero for the wndr3800 and wndr3700v2 is at:

http://snapon.lab.bufferbloat.net/~cero2/cerowrt/= wndr/3.7.1-1/
=0A
=0A

--
Dave T=C3=A4ht
Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subs= cribe.html
_______________________________________________
C= erowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
<= a href=3D"https://lists.bufferbloat.net/listinfo/cerowrt-devel" target=3D"_= blank">https://lists.bufferbloat.net/listinfo/cerowrt-devel
=0A=0A
=0A
=0A=0A=0A=0A=0A<= /div> ------=_20130104160101000000_54591--