From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-x244.google.com (mail-wi0-x244.google.com [IPv6:2a00:1450:400c:c05::244]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 3AD3221F3DF; Fri, 12 Jun 2015 10:55:09 -0700 (PDT) Received: by wibbw19 with SMTP id bw19so4213029wib.2; Fri, 12 Jun 2015 10:55:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=nLscJSJPVrJ2MoT+bkM/FtHjA6vWr0bGRaUVf+GFsyA=; b=dV0cdXV2/3zVSCZF+Lly9m8pIByT5jFnBaOXQ0YIUl/TjkcypfC9t2ZWt74D+BXzsd xp1gjDwFeT/vviusKg8pGASRhQMSrIDthuOr2RyGv5qjqXozB6LPQ+HHd20DldI3Iihv +LMvQsYBa0+mrZ+9UU1BxIaRM2DcaxHX/5osZ7HBebfzAQKUsxbKbR1bKxRvfXoZ43Ej owRx12kQffzf/tXMIj1B4dkdGicBOgWOjRsDbwi/Ma0x1q+ynAjgvN+f4yhRKcidMxze +dt+YsiNihidGoyJCSBhVM9O5sJpRJCkTtJZYbqpTu07OORf/8Ls7haA+3xkPtkdTLQp uhng== X-Received: by 10.180.82.100 with SMTP id h4mr9197165wiy.34.1434131707991; Fri, 12 Jun 2015 10:55:07 -0700 (PDT) Received: from volcano.localdomain (host-89-243-102-33.as13285.net. [89.243.102.33]) by mx.google.com with ESMTPSA id a8sm3764792wic.22.2015.06.12.10.55.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Jun 2015 10:55:06 -0700 (PDT) Message-ID: <557B1CF9.6090401@gmail.com> Date: Fri, 12 Jun 2015 18:55:05 +0100 From: Alan Jenkins User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Daniel Havey References: <5578DEE8.6090209@gmail.com> <2ACEC68A-3795-4F7C-BA0F-FBDBA3732566@gmx.de> <557AD7F1.1000808@gmail.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: cake@lists.bufferbloat.net, "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [Cake] [Bloat] active sensing queue management X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jun 2015 17:55:38 -0000 On 12/06/15 15:35, Daniel Havey wrote: > On Fri, Jun 12, 2015 at 6:00 AM, Alan Jenkins > wrote: >> On 12/06/15 02:44, David Lang wrote: >>> On Thu, 11 Jun 2015, Sebastian Moeller wrote: >>> >>>> On Jun 11, 2015, at 03:05 , Alan Jenkins >>>> wrote: >>>> >>>>> On 10/06/15 21:54, Sebastian Moeller wrote: >>>>> >>>>> One solution would be if ISPs made sure upload is 100% provisioned. >>>>> Could be cheaper than for (the higher rate) download. >>>> >>>> Not going to happen, in my opinion, as economically unfeasible for a >>>> publicly traded ISP. I would settle for that approach as long as the ISP is >>>> willing to fix its provisioning so that oversubscription episodes are >>>> reasonable rare, though. >>> >>> not going to happen on any network, publicly traded or not. >> >> Sure, I'm flailing. Note this was in the context of AQSM as Daniel >> describes it. (Possibly misnamed given it only drops. All the queuing is >> "underneath" AQSM, "in the MAC layer" as the paper says :). >> > Noooooooooo! I am a huge supporter of ECN. ECE Everywhere! I'm sure > I wrote "mark/drop" in the paper. I might have dyslexically written > "drop/mark", but, if I ever gave the impression then I categorically > deny that right now and till forever. ECN everywhere :^) My bad! Imagine I wrote mark/drop. It was just a definitional wibble. Is it AQM if it's not your Q, is it policing (and does AQM include policing)? >> - AQSM isn't distinguishing up/down bloat. When it detects bloat it has to >> limit both directions in equal proportion. >> >> => if there is upload contention (and your user is uploading), you may hurt >> apps sensitive to download bandwidth (streaming video), when you don't need >> to. >> >> What would the solutions look like? >> >> i) If contention in one direction was negligible, you could limit the other >> direction only. Consumer connections are highly asymmetric, and AQSM is >> only measuring the first IP hop. So it's more feasible than 100% in both >> directions. And this isn't about core networks (with larger statistical >> universes... whether that helps or not). >> >> I'm sure you're right and they're not asymmetric _enough_. >> >> >> ii) Sebastian points out if you implement AQSM in the modem (as the paper >> claims :p), you may as well BQL the modem drivers and run AQM. *But that >> doesn't work on ingress* - ingress requires tbf/htb with a set rate - but >> the achievable rate is lower in peak hours. So run AQSM on ingress only! >> Point being that download bloat could be improved without changing the other >> end (CMTS). >> > This is pretty cool. I had not considered BQL (though Dave and Jim > were evangelizing about it at the time :). This solves the > upload/download problem which I was not able to get past in the paper. > BQL on the egress and ASQM for the ingress. BQL will make sure that > the upload is under control so that ASQM can get a good measurement on > the download side. Woot! Woot! Uncooperative ISP problem solved! > > BTW...Why doesn't BQL work on the ingress? Both AQM & BQL manage transmit queues. It's the natural way to work. If you don't control the transmit queue(s) at the bottleneck, you have to insert an artificial bottleneck that you _do_ control. The problem is we don't have the source to hack any modems, only Linux routers like in your paper :). It's why, to set up a router with our SQM, you have to know the line rate (both directions). And tell SQM to use maybe 95% of it. And assume the full rate is available even during peak hours :(. So we still want co-operative ISPs to solve this, because that's who procures the modems. That said, once they've been _developed_ it's going to be easier to buy a good modem regardless of ISP. We rate-limit with htb, so we build a queue for fq_codel to work on. (Then Johnathan smushed the two qdiscs together and called the resulting code "cake"). Regarding ingress rate-limiting, note that - it involves creating a virtual network interface (IFB), to get an artificial transmit queue to apply AQM to. (Something like that, I don't know the exact syntax). - bursts can still pile up at the upstream bottleneck, so you can't eliminate latency increases altogether. (E.g. caused by tcp's initial multiplicative increase). It can still be a big win though, because isp queues would often be allowed to grow many times larger than needed (expected rtt / ~100ms). >>> The question is not "can the theoretical max of all downstream devices >>> exceed the upstream bandwidth" because that answer is going to be "yes" for >>> every network built, LAN or WAN, but rather "does the demand in practice of >>> the combined downstream devices exceed the upstream bandwidth for long >>> enough to be a problem" >>> >>> it's not even a matter of what percentage are they oversubscribed. >>> >>> someone with 100 1.5Mb DSL lines downstream and a 50Mb upstream (30% of >>> theoretical requirements) is probably a lot worse than someone with 100 1G >>> lines downstream and a 10G upstream (10% of theoretical requirements) >>> because it's far less likely that the users of the 1G lines are actually >>> going to saturate them (let alone simultaniously for a noticable timeframe), >>> while it's very likely that the users of the 1.5M DSL lines are going to >>> saturate their lines for extended timeframes. >>> >>> The problem shows up when either usage changes rapidly, or the network >>> operator is not keeping up with required upgrades as gradual usage changes >>> happen (including when they are prevented from upgrading because a peer >>> won't cooperate) >>> >>> As for the "100% provisioning" ideal, think through the theoretical >>> aggregate and realize that before you get past very many layers, you get to >>> a bandwidh requirement that it's not technically possible to provide. >>> >>> David Lang >> > Yuppers! Dave is right. The FCC studies (especially the 80/80 study > out of UNC) from 2010 - 2014 (footnoted in the paper) indicate that > during peak hours it is quite common for an ISP not to provide 100% of > the rated throughput. Note the same technical problem applies if they provide 100%-150%.[1] If bloat is a problem, you can probably still hack around it as a user, but then you lose the _benefits_ of the oversubscription. [1] Comcast "Powerboost" http://www.dslreports.com/faq/14520 > In fact in 2014 it indicates that a full on 50% > of the ISPs measured provided less than 100%. The 100% all the time > goal is unreasonable because it implies too much waste. Many ISPs get > to 90% or above even during peak hours. This is good! We could live > with that :) Providing that last 10% would mean that they would have > to provide for a lot of excess capacity that goes unused during > non-peak hours. Wasteful. That money should be allocated for more > important things like providing AQM for all or saving the planet or > something else. :^) :) I grant Dave's caveats. Couldn't resist looking for the graphs. (Around page 40 http://data.fcc.gov/download/measuring-broadband-america/2014/2014-Fixed-Measuring-Broadband-America-Report.pdf). It looks like cable ISPs happen to hit an average of 100% during peak hours (as you might hope from figure of 50% going below, apparently 50% are going above). Alan