From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 50B6321F221 for ; Tue, 25 Feb 2014 07:55:07 -0800 (PST) Received: by mail-wg0-f45.google.com with SMTP id y10so561101wgg.28 for ; Tue, 25 Feb 2014 07:55:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=mD5Kr2djcEP7GA2mvLeaBOKW1TlqHRE6Iix5y8NqsL0=; b=XroUbyP6DsOZ35BqF40aXOhWskdox77mYoVIArrQHrvSMo9dQ2UL93pAGhnklEn5e2 SAa0dQM0IE0mt8QrvNW6ZKttCEP2/cy+RH1qadhUsxsU3yy7yZoGhXww1bcsd5nPi4Th v1xw+PXdWhr/Hj3xyED0zAHK3CKF8y2Yt/+YnqtNE7lT6FFeEywX7TxVqJbfJ9onIFaZ 2cnkYtiVWQOt+DrQ7USkC1JySyq82wmYNCjCypKOg9TnaJffhYBWRTh+qhiwcY/qJ9mD 71MlulL/ieC/gN3LIebjJOCytk9ntc9HDzV1gFqIVlTaiQwxlnq5LqINliXVhjZK+MUB /pmg== MIME-Version: 1.0 X-Received: by 10.194.90.144 with SMTP id bw16mr24985675wjb.1.1393343699093; Tue, 25 Feb 2014 07:54:59 -0800 (PST) Received: by 10.216.8.1 with HTTP; Tue, 25 Feb 2014 07:54:58 -0800 (PST) In-Reply-To: <34C9F2ED-49BD-43DF-8421-67D1D3CBA5E1@gmx.de> References: <4E5BC321-2054-4364-BECC-DF34E0D20380@gmail.com> <34C9F2ED-49BD-43DF-8421-67D1D3CBA5E1@gmx.de> Date: Tue, 25 Feb 2014 07:54:58 -0800 Message-ID: From: Dave Taht To: Sebastian Moeller Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Stuart Cheshire , cerowrt-devel Subject: Re: [Cerowrt-devel] Equivocal results with using 3.10.28-14 X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 15:55:08 -0000 On Tue, Feb 25, 2014 at 5:37 AM, Sebastian Moeller wrote: > Hi Rich, > > > On Feb 25, 2014, at 14:09 , Rich Brown wrote: > >> Thanks everyone for all the good advice. I will summarize my responses t= o all your notes now, then I'll go away and run more tests. >> >> - Yes, I am using netperf 2.6.0 and netperf-wrapper from Toke's github r= epo. >> >> - The "sync rate" is the speed with which the DSL modem sends bits to/fr= om my house. I got this by going into the modem's admin interface and pokin= g around. (It turns out that I have a very clean line, high SNR, low attenu= ation. I'm much less than a km from the central office.) So actual speed sh= ould approach this, except... > > I would think of this as the theoretical upper limit ;) > >> >> - Of course, I have to subtract all those overheads that Sebastian descr= ibed - ATM 48-in-53, which knocks off 10%; ATM frame overhead which could a= dd up to 47 bytes padding to any packet, etc.) >> >> - I looked at the target calculation in Dave's Home Gateway best practic= es. (http://snapon.lab.bufferbloat.net/~d/draft-taht-home-gateway-best-prac= tices-00.html) Am I correct that it sets the target to five 1500-byte packe= t transmission time or 5 msec, whichever is greater? > > Note, the auto target implementation in ceropackages-3.10 sqm-scr= ipts uses the following: > > adapt_target_to_slow_link() { > CUR_LINK_KBPS=3D$1 > CUR_EXTENDED_TARGET_US=3D > MAX_PAKET_DELAY_IN_US_AT_1KBPS=3D$(( 1000 * 1000 *1540 * 8 / 1000 )) > CUR_EXTENDED_TARGET_US=3D$(( ${MAX_PAKET_DELAY_IN_US_AT_1KBPS} / ${CU= R_LINK_KBPS} )) # note this truncates the decimals > # do not change anything for fast links > [ "$CUR_EXTENDED_TARGET_US" -lt 5000 ] && CUR_EXTENDED_TARGET_US=3D50= 00 > case ${QDISC} in > *codel|pie) > echo "${CUR_EXTENDED_TARGET_US}" > ;; > esac > } > > This is modeled after the shell code Dave sent around, and does not exact= ly match the free version, because I could not make heads and tails out of = the free version. (Happy to discuss change this in SQM if anybody has a bet= ter idea) Really the target calculation doesn't matter so much so long as it's larger than the MTU. This is somewhat an artifact of htb which buffers up an extra packet, and the cpu scheduler... It is becoming clearer (with the recent description of the pie + rate limiter + mac compensation stuff in future cablemodems) that we can do much work to improve the rate limiters, and should probably intertwine them like what the cable folk just did, in order to get best performance. >> >> - I was astonished by the calculation of the bandwidth consumed by acks = in the reverse direction. In a 7mbps/768kbps setting, I'm going to lose one= quarter of the reverse bandwidth? Wow! Yes. TCP's sending ack requirement was actually the main driver for having any bandwidth at all on the upstream. Back in the 90s all cable providers wanted to provide was sufficient bandwidth for a "buy" button. But, due to this behavior of TCP, a ratio between down/up of somewhere between 6 and 12 to 1 was what was needed to make TCP work well. (and even then they tried hard to make it worse and ack compression is still part of many cable modem's provisioning) IPv6 has much larger acks than ipv4.... I did a lot of work on various forms of hybrid network compression back in the day (early 90s), back when we had a single 10Mbit radio feeding hundreds of subscribers, and a 14.4 modem sending back acks... and data. It turned out that a substantial percentage of subscribers actually wanted to upload stuff... and that we couldn't achieve 10Mbit in the lab with a 14.4 return... and that you can only slice up 10Mbit so many ways before you ran out of bandwidth, and you run out of bandwidth long before you can turn a profit... (and while I remember details of the radio setup and all the crazy stuff we did to get more data through the modems, I can't remember the name of the company) At the time I was happy, sorta, in that we'd proven that future ISPs HAD to provide some level of upstream bandwidth bigger than a buy button, and the original e2e internet was going to be at least somewhat preserved... I didn't grok at the time that NAT was going to be widely deployed... I mean, at the time, you sold a connection, and asked how big a network did you want, we defaulted to a /28, and we were still handing out class C's to everyone that asked. > Well, so was I (first time I did that) but here is the kicker wit= h classical non0-delayed ACKs this actually doubles since each data packet = gets acknowledged (I assume this puts a lower bound on how asymmetrical a l= oin an ISP can sell ;) ). But since I figured out that macosx seems to defa= ult to 1 Ack for every 4 packets, so only half the traffic. And note any tr= uly bi-directional traffic shouldbe able to piggyback many of those ACKs in= to normal data packets. I doubt your '4'. Take a capture for a few hundred ms. My understanding of how macos works is that after a stream is sustained for a while, it switches from one ack every 2 packets into "stretch acks" - to one every 6 (or so). there are some interesting bugs associated with stretch acks, and also TSO - in one case we observed a full TSO of TCP RSTs being sent instead of one. >> >> - I wasn't entirely clear how to set the target in the SQM GUI. I believ= e that "target ##msec" is an acceptable format. Is that correct? > > In the new version with the dedicated target field, "40ms" will w= ork as will "40000us", "40 ms". In the most recent version "auto" will set = the auto adjustment of target (it will also extend interval by the new-targ= et - 5ms) to avoid the situation where Target gets larger than interval. > In the older versions you would put "target 40ms" into the egress= advanced option string. Note I used double quotes in my examples for clari= ty, the GUI does not want those... > > >> >> - There's also a discussion of setting the target with "auto", but I'm n= ot sure I understand the syntax. > > just type in auto. You can check with log read and "tc -d qdsic" > > Best Regards > Sebastian > >> >> Now to find some time to go back into the measurement lab! I'll report a= gain when I have more data. Thanks again. >> >> Rich >> >> >> >> On Feb 24, 2014, at 9:56 AM, Aaron Wood wrote: >> >>> Do you have the latest (head) version of netperf and netperf-wrapper? = some changes were made to both that give better UDP results. >>> >>> -Aaron >>> >>> >>> On Mon, Feb 24, 2014 at 3:36 PM, Rich Brown w= rote: >>> >>> CeroWrt 3.10.28-14 is doing a good job of keeping latency low. But... i= t has two other effects: >>> >>> - I don't get the full "7 mbps down, 768 kbps up" as touted by my DSL p= rovider (Fairpoint). In fact, CeroWrt struggles to get above 6.0/0.6 mbps. >>> >>> - When I adjust the SQM parameters to get close to those numbers, I get= increasing levels of packet loss (5-8%) during a concurrent ping test. >>> >>> So my question to the group is whether this behavior makes sense: that = we can have low latency while losing ~10% of the link capacity, or that get= ting close to the link capacity should induce large packet loss... >>> >>> Experimental setup: >>> >>> I'm using a Comtrend 583-U DSL modem, that has a sync rate of 7616 kbps= down, 864 kbps up. Theoretically, I should be able to tell SQM to use numb= ers a bit lower than those values, with an ATM plus header overhead with de= fault settings. >>> >>> I have posted the results of my netperf-wrapper trials at http://richb-= hanover.com - There are a number of RRUL charts, taken with different link = rates configured, and with different link layers. >>> >>> I welcome people's thoughts for other tests/adjustments/etc. >>> >>> Rich Brown >>> Hanover, NH USA >>> >>> PS I did try the 3.10.28-16, but ran into troubles with wifi and ethern= et connectivity. I must have screwed up my local configuration - I was doin= g it quickly - so I rolled back to 3.10.28.14. >>> _______________________________________________ >>> Cerowrt-devel mailing list >>> Cerowrt-devel@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>> >> >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel > > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html