From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id C906721F0BD for ; Fri, 4 Jan 2013 13:44:53 -0800 (PST) Received: by mail-ie0-f180.google.com with SMTP id c10so19948383ieb.39 for ; Fri, 04 Jan 2013 13:44:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=VCVOX07w7WkFedbzDFqTqh+JrWZvG+JwQ3HR1tvXZuE=; b=QWPTt3vnQ05JvmG47KOGQ7fORvCn7bZFvU1UTA7YYwh1Kv8+FskydlPDrHg+Vj2gm9 d8qQGGCofBYw8y/FG9N1ooivVbcQtvJRzfY+c9Abp2E9dPSbMfMbsxXkUOv4r41HBjKV QQ2a08+DIxHBOtpvBvrnD28nqW+IPwUu0TfpJ0H0oIwoYSyMLwbD3D2ZYsPUOJyoJtu7 puxzxz5hWc94ODbI96Ik2RefZvkIly9nKPm+iDYkP0/PfC1lA339ZNflL5iLEwrJF1iq +F6JrWdESeap1KnAJGF8RxNAlDxwK/h5Wazgc8y48gagQXVmNaQt9QYdoHZIOJ0nvce6 a+lg== MIME-Version: 1.0 Received: by 10.50.213.73 with SMTP id nq9mr46344755igc.27.1357335893135; Fri, 04 Jan 2013 13:44:53 -0800 (PST) Received: by 10.64.135.39 with HTTP; Fri, 4 Jan 2013 13:44:53 -0800 (PST) In-Reply-To: References: Date: Fri, 4 Jan 2013 13:44:53 -0800 Message-ID: From: Dave Taht To: Jerry Chu Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Eric Dumazet , cerowrt-devel@lists.bufferbloat.net, Yuchung Cheng Subject: Re: [Cerowrt-devel] TFO crashes cerowrt 3.7.1-1 X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2013 21:44:54 -0000 On Fri, Jan 4, 2013 at 1:36 PM, Jerry Chu wrote: > On Fri, Jan 4, 2013 at 1:21 PM, Dave Taht wrote: >> >> On Fri, Jan 4, 2013 at 12:57 PM, Jerry Chu wrote: >> > +ycheng >> > >> > On Fri, Jan 4, 2013 at 12:43 PM, Maciej Soltysiak >> > wrote: >> >> >> >> Oops, apologies if email was formatted weirdly... >> > >> > >> > The problem you described below is separate from the MIPS router crash >> > one, >> > right? >> >> I think - but am of course unsure - that we've actually hit a >> different (or additional) problem than TFO - merely exposed by testing >> fastopen first... >> >> that said... >> >> >BTW, we've only tested on x86_64 arch. >> >> I have multiple arches accumulated for some BQL work, mostly arm, >> >> They include a raspberry pi, a zedboard, a wndr4300, a nexus 7, and a >> couple other boxes in addition to my usual mips based wndr3800s and >> nanostation m5s. (which use mildly different mips chips and in >> particular the m5 does not share the unaligned rx problem that the >> ar71xx has a patch for). The zedboard's been problematic... >> >> So I will fold in testing tfo to looking hard at the igmp, and >> checksum issues seemingly exposed today. Unless someone beats me to it >> (I'm tied up til sunday) >> >> In the interim, at a higher layers, the current release of httpping >> has tfo support, and there are patches to polipo, that might benefit > > > Awesome! > >> >> from review and wider testing on x86_64. >> >> >> https://raw.github.com/dtaht/ceropackages-3.3/master/net/polipo/patches/= 001-server_tfo.patch > > > tfo_qlen is your first line of defense against spoofed TFO attack so you > want to pick a value wisely. (50 seems on the safer side.) I would like it if TFO client support landed in polipo too, which looked both tricky to implement, and useful as a means of getting more TFO support "out there". Early testing with httping was quite promising over wifi. Eliminating an RTT there basically doubled throughput for short local aggregates. http://www.vanheusden.com/httping/ (it's the -F option) Are there any other test tools available? > > Jerry > >> >> >> At least half of the needed support landed in netperf svn recently, too. >> >> > >> > In addition to tcpdump, "netstat -s | grep -i fastopen" may be useful >> > too. >> > >> > Thanks, >> > >> > Jerry (author of TFO server code) >> > >> >> >> >> >> >> On Fri, Jan 4, 2013 at 9:42 PM, Maciej Soltysiak >> >> wrote: >> >>> >> >>> I am seeing something strange here, with polipo related to TFO but >> >>> also >> >>> DNS. >> >>> When I just took 3.7.1-1 and set my windows 7 laptop to use >> >>> gw.home.lan:8123 as http proxy it didn't work. What I observed was: >> >>> A) after quite a while polipo's response to browser was 504 Host >> >>> www.osnews.com lookup failed: Timeout >> >>> b) this error in ssh console: Host osnews.com lookup failed: Timeout >> >>> (131072) >> >>> c) Disabling TFO by adding option useTCPFastOpen 'false' to config >> >>> 'polipo' 'general' works around the problem >> >>> d) Alternatively, you can keep TFO enabled in polipo but change opti= on >> >>> 'dnsUseGethostbyname' from 'reluctantly' to 'true' (!) >> >>> This is very weird, because TFO is TCP and the DNS queries fired off >> >>> by >> >>> polipo are UDP: >> >>> root@OpenWrt:/tmp/log# tcpdump -n -v -vv -vvv -x -X -s 1500 -i lo >> >>> 20:21:56.160245 IP (tos 0x0, ttl 64, id 50129, offset 0, flags [DF], >> >>> proto UDP (17), length 60) >> >>> 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd17f!] >> >>> 55396+ >> >>> A? www.osnews.com. (32) >> >>> 0x0000: 4500 003c c3d1 4000 4011 78dd 7f00 0001 E..<..@.@.x..... >> >>> 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d.. >> >>> 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn >> >>> 0x0030: 6577 7303 636f 6d00 0001 0001 ews.com..... >> >>> 20:21:56.160319 IP (tos 0x0, ttl 64, id 50130, offset 0, flags [DF], >> >>> proto UDP (17), length 60) >> >>> 127.0.0.1.47304 > 127.0.0.1.53: [bad udp cksum 0xfe3b -> 0xd164!] >> >>> 55396+ >> >>> AAAA? www.osnews.com. (32) >> >>> 0x0000: 4500 003c c3d2 4000 4011 78dc 7f00 0001 E..<..@.@.x..... >> >>> 0x0010: 7f00 0001 b8c8 0035 0028 fe3b d864 0100 .......5.(.;.d.. >> >>> 0x0020: 0001 0000 0000 0000 0377 7777 066f 736e .........www.osn >> >>> 0x0030: 6577 7303 636f 6d00 001c 0001 ews.com..... >> >>> 20:21:56.169942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], pro= to >> >>> UDP (17), length 123) >> >>> 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe7a -> 0x5f73!] >> >>> 55396 >> >>> q: A? www.osnews.com. 1/2/0 www.osnews.com. [29m3s] A 74.86.31.159 n= s: >> >>> osnews.com. [29m3s] NS ns2.swelter.net., osnews.com. [29m3s] NS >> >>> ns1.swelter.net. (95) >> >>> 0x0000: 4500 007b 0000 4000 4011 3c70 7f00 0001 E..{..@.@.> >>> 0x0010: 7f00 0001 0035 b8c8 0067 fe7a d864 8180 .....5...g.z.d.. >> >>> 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn >> >>> 0x0030: 6577 7303 636f 6d00 0001 0001 c00c 0001 ews.com......... >> >>> 0x0040: 0001 0000 06cf 0004 4a56 1f9f c010 0002 ........JV...... >> >>> 0x0050: 0001 0000 06cf 0011 036e 7332 0773 7765 .........ns2.swe >> >>> 0x0060: 6c74 6572 036e 6574 00c0 1000 0200 0100 lter.net........ >> >>> 0x0070: 0006 cf00 0603 6e73 31c0 40 ......ns1.@ >> >>> 20:21:56.173901 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], pro= to >> >>> UDP (17), length 135) >> >>> 127.0.0.1.53 > 127.0.0.1.47304: [bad udp cksum 0xfe86 -> 0x8ecb!] >> >>> 55396 >> >>> q: AAAA? www.osnews.com. 1/2/0 www.osnews.com. [54m44s] AAAA >> >>> 2607:f0d0:1002:62::3 ns: osnews.com. [29m3s] NS ns1.swelter.net., >> >>> osnews.com. [29m3s] NS ns2.swelter.net. (107) >> >>> 0x0000: 4500 0087 0000 4000 4011 3c64 7f00 0001 E.....@.@.> >>> 0x0010: 7f00 0001 0035 b8c8 0073 fe86 d864 8180 .....5...s...d.. >> >>> 0x0020: 0001 0001 0002 0000 0377 7777 066f 736e .........www.osn >> >>> 0x0030: 6577 7303 636f 6d00 001c 0001 c00c 001c ews.com......... >> >>> 0x0040: 0001 0000 0cd4 0010 2607 f0d0 1002 0062 ........&......b >> >>> 0x0050: 0000 0000 0000 0003 c010 0002 0001 0000 ................ >> >>> 0x0060: 06cf 0011 036e 7331 0773 7765 6c74 6572 .....ns1.swelter >> >>> 0x0070: 036e 6574 00c0 1000 0200 0100 0006 cf00 .net............ >> >>> 0x0080: 0603 6e73 32c0 4c ..ns2.L >> >>> This is the only DNS traffic I saw during the attempts. The tcpdumps >> >>> have >> >>> udp bad checksum but when I disabled TFO in polipo, the UDP where >> >>> still bad >> >>> checksum but they worked. >> >>> Really weird. >> >>> p.s. UPNP still works for port forwarding negotiation as it did in >> >>> 3.6.11-4 >> >>> I still couldn't get the UPNP/SSDP broadcasts (udp to 239.255.255.25= 0) >> >>> to >> >>> being forwarded between se00 and sw00/sw10. Last time it worked was >> >>> ~3.3.8. >> >>> I'm starting not to question why it doesn't work, I'm starting to >> >>> wonder why >> >>> it did work then ;-) >> >>> Regards, >> >>> Maciej >> >>> On Fri, Jan 4, 2013 at 6:33 PM, Dave Taht wrot= e: >> >>>> >> >>>> On Fri, Jan 4, 2013 at 9:27 AM, Eric Dumazet >> >>>> wrote: >> >>>> > Sorry, could you give us a copy of the panic stack trace ? >> >>>> >> >>>> I will get a serial console up on a wndr3800 by sunday. (sorry, jus= t >> >>>> landed in california, am in disarray) >> >>>> >> >>>> The latest dev build of cero for the wndr3800 and wndr3700v2 is at: >> >>>> >> >>>> http://snapon.lab.bufferbloat.net/~cero2/cerowrt/wndr/3.7.1-1/ >> >>>> >> >>>> -- >> >>>> Dave T=E4ht >> >>>> >> >>>> Fixing bufferbloat with cerowrt: >> >>>> http://www.teklibre.com/cerowrt/subscribe.html >> >>>> _______________________________________________ >> >>>> Cerowrt-devel mailing list >> >>>> Cerowrt-devel@lists.bufferbloat.net >> >>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >> >>> >> >>> >> >> >> > >> >> >> >> -- >> Dave T=E4ht >> >> Fixing bufferbloat with cerowrt: >> http://www.teklibre.com/cerowrt/subscribe.html > > --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html