From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 48E6C21F1B1 for ; Thu, 8 Aug 2013 03:26:09 -0700 (PDT) Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 17795208C2; Thu, 8 Aug 2013 06:26:08 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Thu, 08 Aug 2013 06:26:08 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=imap.cc; h= content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; s=mesmtp; bh=JpAU30qRAaVigU+llMdDla+6qHE=; b=vtgMxOar5aWsjhlOiP+Ju28/W4n7 WQS76FhoA1zEXulxIdCg4EUbHoaFwDhxaFXuCwUYoz0AymV3OFnUMt9DqYhtgRWS /1wQLtcD+S0I/gJ/BCuBI/5IlJ/m7irk+ypp1WLyhq7d0TmYWSLU2p5JziKK/BKC l8qDBGNZlBKPfok= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id :references:to; s=smtpout; bh=JpAU30qRAaVigU+llMdDla+6qHE=; b=N1 D3J0Nn0M4sbuNBX9zbNpExs34ZWDdLxW/9L44QKLQGDV8e7dAumgnNcBD25kuMcQ 8nDtTAdQKPQ06e7wqOS+0YX44i9o97h/pIqs7ztFksASAwgXelhSg5Q/0dpfepk9 SkkZWZrpfbDygOdgWj96YyZLlA1/XuvE70Ni3zl9U= X-Sasl-enc: RtiaBVxVX89CcAGDLJY9hKdj826uUZBidEM3jmEVxGEl 1375957567 Received: from [172.30.42.15] (unknown [188.221.232.223]) by mail.messagingengine.com (Postfix) with ESMTPA id 6B6A6680107 for ; Thu, 8 Aug 2013 06:26:07 -0400 (EDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) From: Fred Stratton In-Reply-To: Date: Thu, 8 Aug 2013 11:26:06 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <46820144-E1DD-4858-A2B6-38F95D2894DB@imap.cc> <1AEAC257-ACFB-463E-B5CF-C7D086EE5B03@imap.cc> <32269B74-55A6-4CFE-8372-9935325BA774@imap.cc> To: "cerowrt-devel@lists.bufferbloat.net" X-Mailer: Apple Mail (2.1508) Subject: Re: [Cerowrt-devel] cerowrt-3.10.2-1 dev release + owamp X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Aug 2013 10:26:09 -0000 On 8 Aug 2013, at 10:41, Sebastian Moeller wrote: > Hi Fred, >=20 >=20 > On Aug 8, 2013, at 01:21 , Fred Stratton wrote: >=20 >>=20 >> On 7 Aug 2013, at 14:38, Sebastian Moeller wrote: >>=20 >>> Hi Fred, >>>=20 >>> this got a bit longish so I took the liberty to reduce the quoted = text a bit >>>=20 >>>=20 >>> On Aug 5, 2013, at 12:47 , Fred Stratton = wrote: >>>=20 >>> [snipp] >>>=20 >>>>>>> You are using 2 routers in series. I have disabled all routing = functions on the 2wire. It is transparent to the network. >>>>>>=20 >>>>>> Which is exactly the situation I faced with the cable modem = before; my cerowrt-router was provisioned with an IP address through the = bridged cable-modem via DHCP, but I still could access the modem's = 192.168.100.1 with out any configuration required. I know there is some = openwork information = (http://wiki.openwrt.org/doc/howto/access.modem.through.nat) that makes = it look like one needs to do more involved fiddling with the firewall, = but that turned out not to be required with cerowrt. I do not know how = that works if one runs a pppoe client on cerowrt though and I left = cerowrt's ip address assignment in place. (My hunch is that since = cerowrt leaves the typical 192.168.N.N ranges alone the whole issue gets = reduced to a simple routing issue=85 and since Dave takes care that cero = works well as secondary (test) router in a typical home situation, I = guess routing 192.168.N.N is well with in cerowrt's scope) >>>>>> But, I guess you tried that already and it still does not work. = Would be interesting to learn why=85 >>>>>=20 >>>>> The difference is that you have the ISP gateway as a primary = device issuing a DHCP address to the cerowrt secondary router. The 2 = devices are then obviously on the same ipv4 subnet. >>>>>=20 >>>>> I use the 2700 transparently. DHCP is turned off. If I turn it on, = I have to use the device in DMZ mode with its firewall on, which I do = not want to do. >>>=20 >>> Sorry, to keep harping on this, but this is pretty close to what = I did with the cable modem. As I said I had it working with a similar = setup as you have, cerowrt was assigned a public IP (75.142.58.156) = address by the cable-ISPs dhcp server while the modems configuration = interface was running on the "private" 192.168.100.1. So the modem and = cero were decidedly not on the same IP subnet, but still I could connect = to it without needing to change anything. Initially, before I found out = that it works out of the box I had defined an alias IP address on the = wan interface of (WAN2CABLEMODEM ipv4-address: 192.168.100.2; = ipv4-netmask: 255.255.255.0). But it turned out that this was not = necessary as of cerowrt 3.3.8-17 I did not need to do this any more, = accessing the cable modem just worked by directing a browser to = 192.168.100.1. >>>=20 >>> So, have you tried to access the modem recently by simply = directing a browser to its address? And have you tried the same after = just configuring an alias as hinted above? If so what was the result? >>>=20 >>=20 >> I configured an alias using uci at your prompting. It works. I can = now access the 2700. >=20 > Excellent, that is solved then. On to the next issues... >=20 >>=20 >>=20 >>>>>=20 >>>>> Initially, I used the 2700 with the tomatoUSB router attached to = that, and then a router running openWRT. This setup allowed access to = the 2700, through a masquerade in tomatoUSB.=20 >>>>>=20 >>>>> Although ipv6 addresses were propagated throughout the network by = Barrier Breaker, ipv6 did not work, probably because of the way radvd = works in tomato. >>>>>=20 >>>>> I have never used the cerowrt as a secondary device because of = this. >>>>>>=20 >>>>>=20 >>>=20 >>> [snipp] >>>=20 >>>>> I do not want to use cable, which is expensive. The DOCSIS box - a = custom Netgear device - has a poor reputation. >>>>>=20 >>>>> I do not want to use fibre, again, because when it comes here, it = will be supplied by BT/, and is traffic shaped and capped. The BT web = site has 35 pages of price increases for this year. >>>>>=20 >>>>> I will continue with ADSL2+ >>>=20 >>> I fully agree that getting ADSL(2+) links debated and offering = low latency internet access is worthwhile as a considerable number of = people simply have no other choice available. And this is why your case = is so interesting! If you can improve the "interactivity" in your home = and document the required steps somewhere others will have an easier = time. >>>=20 >>> Yes. Hopefully others using ADSL will also participate. >=20 >=20 > Okay, let's assume for a minute that the ATM link layer = adaptation mechanism might be busted. Since each package (and its = overhead) always consume an integer number of ATM cells and no ATM cells = are shared between packets, the worst case ATM overhead would be close = to 50% (a small packet with 48 + 1 byte length will take 2 full 48 byte = ATM cells, just looking at the payload here). For bigger packets the ATM = quantization overhead will be a smaller percentage. So if you shape down = to 50% of link rates (and do your testing not only with very small = packets) this should make the shaping robust even with a busted ATM link = layer adaptation mechanism. Assuming sane numbers of "channels" reserved = for FEC this should also nicely cover the useable link rate reduction = caused by these channels. > So I would like to ask you to try shaping up and down rates to = 50% of the link rates and then see how the link behaves in face of = concurrent streaming and interactive traffic. In case you go that route, = please let us know whether this improves anything or not. Oh, and I = guess it would be a good idea to do these tests not over any tunnel by = by "plain" IP4 or IP6, depending on your actual connection to your ISP. > One other thing to try would be to see whether latencies change = over the course of the day or not (if the uplink bottleneck should be = the uplink of the DSLAM itself shaping on your end will not help much) The ISP provides ipv4 only. ipv6 is from HE and therefore tunnelled, but = sometimes faster on the available speediest. I shall try your = suggestions.=20 >=20 >=20 > Best > Sebastian >=20 >=20 >>>=20 >>> best >>> Sebastian >>>=20 >>>=20 >>>>>>=20 >>>>>> Best >>>>>> Sebastian >>>>>>=20 >>>>>>=20 >>>>>>>>=20 >>>>>>>> best regards >>>>>>>> Sebastian >>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On 3 Aug 2013, at 10:38, Sebastian Moeller = wrote: >>>>>>>>>=20 >>>>>>>>>> Hi Fred, >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> On Aug 1, 2013, at 00:35 , Fred Stratton = wrote: >>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> On 31 Jul 2013, at 23:14, Sebastian Moeller = wrote: >>>>>>>>>>>=20 >>>>>>>>>>>> Hi Fred, >>>>>>>>>>>>=20 >>>>>>>>>>>> thanks a lot. >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> On Jul 31, 2013, at 23:37 , Fred Stratton = wrote: >>>>>>>>>>>>=20 >>>>>>>>>>>>> tc -s -d class show dev ge00 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> class htb 1:10 parent 1:1 leaf 110: prio 0 quantum 1500 = rate 700000bit ceil 700000bit burst 1599b/1 mpu 0b overhead 0b cburst = 1599b/1 mpu 0b overhead 0b level 0=20 >>>>>>>>>>>>> Sent 15809014 bytes 115190 pkt (dropped 4733, overlimits 0 = requeues 0)=20 >>>>>>>>>>>>> rate 3616bit 3pps backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> lended: 115190 borrowed: 0 giants: 0 >>>>>>>>>>>>> tokens: 263560 ctokens: 263560 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> class htb 1:1 root rate 700000bit ceil 700000bit burst = 1599b/1 mpu 0b overhead 0b cburst 1599b/1 mpu 0b overhead 0b level 7=20 >>>>>>>>>>>>> Sent 15809014 bytes 115190 pkt (dropped 0, overlimits 0 = requeues 0)=20 >>>>>>>>>>>>> rate 3616bit 3pps backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> lended: 0 borrowed: 0 giants: 0 >>>>>>>>>>>>> tokens: 263560 ctokens: 263560 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> class fq_codel 110:1b8 parent 110:=20 >>>>>>>>>>>>> (dropped 0, overlimits 0 requeues 0)=20 >>>>>>>>>>>>> backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> deficit 84 count 0 lastcount 0 delay 10us >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> tc -s -d class show dev ifb0 >>>>>>>>>>>>> class htb 1:10 parent 1:1 leaf 110: prio 0 quantum 1500 = rate 7000Kbit ceil 7000Kbit burst 1598b/1 mpu 0b overhead 0b cburst = 1598b/1 mpu 0b overhead 0b level 0=20 >>>>>>>>>>>>> Sent 192992612 bytes 168503 pkt (dropped 0, overlimits 0 = requeues 0)=20 >>>>>>>>>>>>> rate 17096bit 4pps backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> lended: 168503 borrowed: 0 giants: 0 >>>>>>>>>>>>> tokens: 27454 ctokens: 27454 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> class htb 1:1 root rate 7000Kbit ceil 7000Kbit burst = 1598b/1 mpu 0b overhead 0b cburst 1598b/1 mpu 0b overhead 0b level 7=20 >>>>>>>>>>>>> Sent 192992612 bytes 168503 pkt (dropped 0, overlimits 0 = requeues 0)=20 >>>>>>>>>>>>> rate 17096bit 4pps backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> lended: 0 borrowed: 0 giants: 0 >>>>>>>>>>>>> tokens: 27454 ctokens: 27454 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> class fq_codel 110:cc parent 110:=20 >>>>>>>>>>>>> (dropped 10, overlimits 0 requeues 0)=20 >>>>>>>>>>>>> backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> deficit -198 count 1 lastcount 1 ldelay 2.3ms >>>>>>>>>>>>> class fq_codel 110:1d9 parent 110:=20 >>>>>>>>>>>>> (dropped 0, overlimits 0 requeues 0)=20 >>>>>>>>>>>>> backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> deficit 226 count 0 lastcount 0 ldelay 2us >>>>>>>>>>>>> class fq_codel 110:1de parent 110:=20 >>>>>>>>>>>>> (dropped 0, overlimits 0 requeues 0)=20 >>>>>>>>>>>>> backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> deficit 238 count 0 lastcount 0 ldelay 10us >>>>>>>>>>>>> class fq_codel 110:345 parent 110:=20 >>>>>>>>>>>>> (dropped 0, overlimits 0 requeues 0)=20 >>>>>>>>>>>>> backlog 0b 0p requeues 0=20 >>>>>>>>>>>>> deficit 226 count 0 lastcount 0 delay 9us >>>>>>>>>>>>>=20 >>>>>>>>>>>>> I changed the hard coded values in = /usr/lib/aqm/functions.sh to arbitrary values, rebooted and obtained the = same results. Both reflect the 7000kbit/s down and 700kbit/s up I = entered in the window. >>>>>>>>>>>>=20 >>>>>>>>>>>> What is the line rate as read out from the del modem or = specified in your contract? >>>>>>>>>>>=20 >>>>>>>>>>> Speedtest.net shows the rate as circa 8.7 megabits/s down, 1 = megabit/s up. Line has radio frequency interference from unidentified = sources.. >>>>>>>>>>=20 >>>>>>>>>> So it looks like specify a generous reserve for the = shaper. Can you log into your modem and get the current line rates? The = rf interference, is it constant (if you can get nice SNR per sub carrier = or even ust bit loading per frequency plots) that is does it only affect = the same frequencies or does it change? (I ask, because temporary = interference might reduce the effective line rate, potentially moving = the buffer back into the del modem) >>>>>>>>>>=20 >>>>>>>>>>> Target snr upped to 12 deciBel. Line can sustain 10 = megabits/s with repeated loss of sync.at lower snr. Contract is for 'up = to 20megabits/s'. =20 >>>>>>>>>>=20 >>>>>>>>>> So ADSL2+ as you even mentioned it before. >>>>>>>>>>=20 >>>>>>>>>>> 850 metres from exchange. Line length circa 1.2km. >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>> I ticked the adsl box. Altering the value in functions.sh = and unticking the box, with reboot, produced the same outcome. >>>>>>>>>>>>=20 >>>>>>>>>>>> This nicely shows I screwed up my testing (and or forgot = to reboot between changes). Or I did try too high a data rate (initially = 97% of the raw link rate) >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> traceroute google.com >>>>>>>>>>>>> traceroute: Warning: google.com has multiple addresses; = using 173.194.41.128 >>>>>>>>>>>>> traceroute to google.com (173.194.41.128), 64 hops max, 52 = byte packets >>>>>>>>>>>>> 1 172.30.42.1 (172.30.42.1) 0.631 ms 0.323 ms 0.249 ms >>>>>>>>>>>>> 2 * * * >>>>>>>>>>>>> 3 10.1.3.234 (10.1.3.234) 22.596 ms 21.241 ms 22.392 = ms >>>>>>>>>>>>> 4 * 10.1.3.214 (10.1.3.214) 27.018 ms 26.703 ms >>>>>>>>>>>>> 5 10.1.4.249 (10.1.4.249) 29.682 ms 28.923 ms 27.479 = ms >>>>>>>>>>>>> 6 * * * >>>>>>>>>>>>> 7 * 209.85.252.186 (209.85.252.186) 30.379 ms * >>>>>>>>>>>>> 8 72.14.238.55 (72.14.238.55) 25.745 ms 25.345 ms = 25.594 ms >>>>>>>>>>>>> 9 lhr08s03-in-f0.1e100.net (173.194.41.128) 27.566 ms = 27.390 ms 27.663 ms >>>>>>>>>>>>>=20 >>>>>>>>>>>>> mtr shows packet losses at hops 2-5=20 >>>>>>>>>>>>> 10.1.3.* are Internet Watch Foundation. >>>>>>>>>>>>=20 >>>>>>>>>>>> This looks pretty reasonable for an adsl link (could be = way worse with higher interleaving) >>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> Netalyzr was used. I appreciate it is an imperfect metric. >>>>>>>>>>>=20 >>>>>>>>>>> OK. Like the ping train idea. Cannot get netperf 2.6.0 to = build on Ubuntu 12.04 >>>>>>>>>>=20 >>>>>>>>>> So I typically run a 1000 count ping against the nearest = host that is on the other side of the DSL link that also gives = consistent ping RTTs without load. Then I start my test loads like = saturating the upload with a long runnig TCP transfer and opening 99 = media heavy tabs in a browser (I really should try the chrome benchmark = that Dave is using). And the I simply look through the ping statistic = results, typically I look at the maximum, and at the standard deviation = to get a handle on how tight the shaper held latency under control. (If = I should get netperf-wrapper to work under Macosx I will try to use that = for testing, but it does not even install, and if I get past that hurdle = I will have to adjust for the differences between Gnu ping and BSD = ping). >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> Best Regards >>>>>>>>>> Sebastian >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> Well, I ran into this issue before. In short netalyzr's = worst case delay numbers do not seem to reflect how an fq_codelled = connection feels. =20 >>>>>>>>>>>> Netalyzr uses an unresponsive UDP probe to force the = bottleneck router's buffers to fill up; with unresponsiveness being a = property no sane flow over the intent should exhibit. Codel/fq_codel is = tailored for responsive flows and will only gradually increase its drop = frequency so responsive TCP flows will be controlled gently and keep = link utilization high. Given enough time codel will also rein in an = unresponsive flows. But netalyzr's probe duration is too short for that = to be happening during netalyzr's runtime. >>>>>>>>>>>> Fq_codel in my experience does a decent job at keeping = interactivity high even with competing traffic like netalyzr (so turn a = ping train against say 10.1.3.234 while netalyzr runs or try = netperf-wrapper in addition).=20 >>>>>>>>>>>> So netalyzr really probes the worst case buffer depth = against basically a "denial of service" type of load; I am not fully = sure what the expectancy on the disc here should be. >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>> best >>>>>>>>>>>> Sebastian >>>>>>>>>>>>=20 >>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> On 31 Jul 2013, at 21:38, Sebastian Moeller = wrote: >>>>>>>>>>>>>=20 >>>>>>>>>>>>>> tc -s -d class show dev ge00 >>>>>>>>>>>>>=20 >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Cerowrt-devel mailing list >>>>>>>>>>>>> Cerowrt-devel@lists.bufferbloat.net >>>>>>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Cerowrt-devel mailing list >>>>>>>>>>> Cerowrt-devel@lists.bufferbloat.net >>>>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> _______________________________________________ >>>>>>>>> Cerowrt-devel mailing list >>>>>>>>> Cerowrt-devel@lists.bufferbloat.net >>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>>>>>>>=20 >>>>>>>=20 >>>>>>> _______________________________________________ >>>>>>> Cerowrt-devel mailing list >>>>>>> Cerowrt-devel@lists.bufferbloat.net >>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>>>>>=20 >>>>>=20 >>>>=20 >>>> _______________________________________________ >>>> Cerowrt-devel mailing list >>>> Cerowrt-devel@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>>=20 >>=20 >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel >=20