From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id B8BD321F1F4; Thu, 11 Jun 2015 00:58:44 -0700 (PDT) Received: from u-089-d065.biologie.uni-tuebingen.de ([134.2.89.65]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MU0U9-1YcJiC0SOX-00QmsX; Thu, 11 Jun 2015 09:58:41 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <5578DEE8.6090209@gmail.com> Date: Thu, 11 Jun 2015 09:58:39 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5578DEE8.6090209@gmail.com> To: Alan Jenkins X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:kRSOjaJpF41xPhHAylZpMm8RCdb2Nn2GKgjkZgRP+YDpX4XfAnM SbgzP/LQz0nbnsD2Dy+VFod+m145jQFK7stw4e71acEh/CCdyi0lSFbAm7nV1cHBLEzRci2 gkKcwplk32ofy6Fr2f1xzxMhMWp1d/U0pRzbYv130dAFF9rFOWZ2s13sHBf91YQrb//8v8F zFHOqbAFHdlJ7fBI7Mz8w== X-UI-Out-Filterresults: notjunk:1; Cc: cake@lists.bufferbloat.net, Daniel Havey , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [Cake] active sensing queue management X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jun 2015 07:59:13 -0000 Hi Alan, On Jun 11, 2015, at 03:05 , Alan Jenkins = wrote: > On 10/06/15 21:54, Sebastian Moeller wrote: >> Hi Dave, >>=20 >>=20 >> On Jun 10, 2015, at 21:53 , Dave Taht wrote: >>=20 >>> http://dl.ifip.org/db/conf/networking/networking2015/1570064417.pdf >>>=20 >>> gargoyle's qos system follows a similar approach, using htb + sfq, = and >>> a short ttl udp flow. >>>=20 >>> Doing this sort of measured, then floating the rate control with >>> "cake" would be fairly easy (although it tends to be a bit more >>> compute intensive not being on a fast path) >>>=20 >>> What is sort of missing here is trying to figure out which side of = the >>> bottleneck is the bottleneck (up or down). >> Yeah, they relay on having a reliable packet reflector upstream = of the =93bottleneck=94 so they get their timestamped probe packets = returned. >=20 > They copy & frob real IP headers. They don't _say_ how the reflection = works, but I guess low TTL -> ICMP TTL exceeded, like traceroute. Then = I read Gargoyle also use ICMP TTL exceeded and I thought my guess is = quite educated 8). Daniel elucidated their magic packets: they create = self-addressed IP packets at the simulated CPE and inject them in the = simulated cable link; the other end will pass the data through its stack = and once the sender-self-addressed packet reaches the IP-layer of the = simulated CMTS it gets send back, since that IP layer sees the CPE=92s = IP address as the to address. @Daniel, this trick can only work if a) the magic packets are = only passed one IP-hop since the first upstream IP-layer will = effectively bounce them back (so the injector in the docsis case needs = to be the cable modem) b) the CPE actually has an IP that can be reached = from the outside and that is known to the person setting up your AQM, is = that correct? How does this work if the CPE acts as an ethernet bridge = without an external IP? >=20 > Note the size of the timestamp, a generous 8 bytes. It "just happens" = that ICMP responses are required to include the first 8 bytes of the IP = payload 8). >=20 >> In the paper they used either uplink or downlink traffic so figuring = where the bottleneck was easy at least this is how I interpret = =93Experiments were performed in the upload (data flowing from the users = to the CDNs) as well as in the download direction.". At least this is = what I get from their short description in glossing over the paper. >=20 > Ow! I hadn't noticed that. You could reduce both rates = proportionally but the effect is inelegant. I think that it what they do, as long as one only measures = uni-directional saturating traffic this approach will work fine as the = bandwidth loss in the opposite direction simply does not materialize. > I wonder what Gargoyle does... >=20 > 2012 gargoyle developer comment says "There are not settings for = active congestion control on the uplink side. ACC concentrats on the = download side only." >=20 > Random blog post points out this is sufficient to fix prioritization = v.s. bufferbloat. "In upstream direction this is not a big problem = because your router can still prioritize which packet should be sent = first". (Yay, I get to classify every application I care about /s and = still get killed by uploads in http). Not fully convinced that this is fully sane, as in cable systems = the upstream bandwidth can fluctuate significantly depending on how many = people are active. Actually scratch the =93cable=94 since most customer = links have shared oversubscribed links somewhere between the CPE and the = internet that will make static bandwidth shaping mis-behave some of the = time. A good ISP just manages the oversubscription well enough that this = issue only occurs transiently=85 (I hope). >=20 > One solution would be if ISPs made sure upload is 100% provisioned. = Could be cheaper than for (the higher rate) download. Not going to happen, in my opinion, as economically unfeasible = for a publicly traded ISP. I would settle for that approach as long as = the ISP is willing to fix its provisioning so that oversubscription = episodes are reasonable rare, though. >=20 >> Nice paper, but really not a full solution either. Unless the = ISPs cooperate in supplying stable reflectors powerful enough to support = all downstream customers. >=20 > I think that's a valid concern. Is "TTL Exceeded" rate-limited like = Echo (because it may be generated outside the highest-speed forwarding = path?), and would this work as tested if everyone did it? I thing Daniel agrees and that is why they came up with the = =93magic=94 packet approach (that drags in its own set of challenges as = far as I can see). >=20 >> But if the ISPs cooperate, I would guess, they could eradicate = downstream buffer bloat to begin with. Or the ISPs could have the = reflector also add its own UTC time stamp which would allow to dissect = the RTT into its constituting one-way delays to detect the currently = bloated direction. (Think ICMP type 13/14 message pairs "on steroids", = with higher resolution than milliseconds, but for buffer bloat detection = ms resolution would probably be sufficient anyways). Currently, I hear = that ISP equipment will not treat ICMP requests with priority though. >> Also I am confused what they actually simulated: =93The modems = and CMTS were equipped with ASQM, CoDel and PIE,=94 and =93However, the = problem pop- ularly called bufferbloat can move about among many queues = some of which are resistant to traditional AQM such as Layer 2 MAC = protocols used in cable/DSL links. We call this problem bufferbloat = displacement.=94 seem to be slightly at odds. If modems and CTMS have = decent AQMs all they need to do is not stuff their sub-IP layer = queuesand be done with it. The way I understood the cable labs PIE = story, they intended to do exactly that, so at least the =93buffer = displacement=94 remedy by ASQM reads a bit like a straw man argument. = But as I am a) not of the cs field, and b) only glossed over the paper, = most likely I am missing something important that is clearly in the = paper... >>=20 >> Best Regards >> Sebastian >=20 > I had your reaction about pie on the modem. >=20 > We could say there is likely room for improvement in any paper, that = claims bufferbloat eliminated with a "target" parameter of 100ms :p. = Results don't look that bad (why?) but I do see 25ms bloat v.s. = codel/pie. It may be inevitable but deserves not to be glossed over = with comparisons to the unrelated 100ms default parameter of codel, = which in reality is the one called "interval" not "target" :). Good QM = on the modem+cmts has got to be the best solution. I fully agree. I have a hunch that their method might be used to = supplement docsis 3.1 pie so that the CPEs can also meaningfully measure = and control downstream buffer bloat in addition to the upstream without = the need to fix the CMTSs. As far as I understand cable labs are quite = proactive in trying to fix this in CPE=92s while I have heard nothing = about the CMTS manufacturers=92 plans (I think the Arris paper was about = CPEs not CMTS). Maybe cable labs could be convinced to try this in = addition to upstream PIE as a solution that will require no CMTS = involvement=85 (I simple assume that the CMTS does not need to = cooperate, but note that the paper seems to rely totally on simulated = data, in so far as linux pc=92s where used to model each of the network = components. So "no real CMTS was harmed during the making of this = paper") Best Regards Sebastian >=20 > Alan