From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-x22e.google.com (mail-lf0-x22e.google.com [IPv6:2a00:1450:4010:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D08E63B2A3 for ; Sat, 24 Dec 2016 10:55:51 -0500 (EST) Received: by mail-lf0-x22e.google.com with SMTP id t196so153802089lff.3 for ; Sat, 24 Dec 2016 07:55:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0Ap4xkyWCZHiuJY+eyp4r13WyPNk5QrNobwdB2q+qZ0=; b=SNPQ7OgmeOkRUSAGDt1O+kfuTgej+BK8QVhSLr120EHO4RNqmZGCi07ntYXkxDWiOa 6ypp2IJtr7rISeNCTB+N9NIgA695SFT6zJ4k2Oi8yZcrwj0GjoJiCbrYpOpB/h6hNPW1 iab5pJE0RpoHBBPgftqRkKeUyISxbgCglym+sylFpuR1aQVnL94h3X4Ph5Z7qMyEixKg Rb72zrtbSNCflnUM1LqL0y50nzMlEpMjb/W3k434fsdgHBRq/+oxMlGWPNYVHTJ2O9SU mgziuBWP5DSF2gasyLAy7z+9Tdy7sAvHT1wF3rvcWyqxGNXGjdO4P99Z66ToFtvGcGoT 9Mlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0Ap4xkyWCZHiuJY+eyp4r13WyPNk5QrNobwdB2q+qZ0=; b=YTnVMJvXP+4EyH9RG14VPxWkbBAZ0l5NJwzdkeSN6MwJYuq6p+9RWPKho08wnv3tnp AyCex3uS23uuZtx+yS+1tB2+vQGkUJTvx7k4QGCijEl4GlMmQrsEJ309udAntEq6ILV3 WxnsTMEmACb+NuFHBPAD5arUmRgzHfM/GNWKFfNYPJ2/gzkuNohsdkXZZVwejvYLUeSy Yhmpz/MUDpSL//W/Pn9x1xA3t9XmgwI6yNHeBLZEpggvSvcTJnCR4CVHZKQiXhV3nsTx ODj47NNKICeZsO3gmo9DZHULcIFVGc1z6Hh43k2dgqGnX8IJLV69Q9/zvRMScxUUw5vS uQQg== X-Gm-Message-State: AIkVDXKRPyP0v/EXB3urKI819hZ+Jco/0PWh9qrkj0kF/EZH9VHMg5fc7cPKIUYJM5BTHy8MnT5qHWcV9/lWAA== X-Received: by 10.25.215.212 with SMTP id q81mr7428100lfi.126.1482594950511; Sat, 24 Dec 2016 07:55:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.198.212 with HTTP; Sat, 24 Dec 2016 07:55:49 -0800 (PST) In-Reply-To: References: <20161222174349.5bd5dffd@xeon-e3> <9A8ACB3B-8411-414E-B2C3-ABE2C276B351@gmail.com> <606D54F6-59FB-49D6-A969-F26434EF9292@gmx.de> From: Benjamin Cronce Date: Sat, 24 Dec 2016 09:55:49 -0600 Message-ID: To: Jonathan Morton Cc: Sebastian Moeller , cake@lists.bufferbloat.net Content-Type: multipart/alternative; boundary=001a11412c7a5f0497054469890b Subject: Re: [Cake] upstreaming cake in 2017? X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2016 15:55:52 -0000 --001a11412c7a5f0497054469890b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Fri, Dec 23, 2016 at 3:53 AM, Jonathan Morton wrote: > >> As far as Diffserv is concerned, I explicitly assume that the standard > RFC-defined DSCPs and PHBs are in use, which obviates any concerns about > Diffserv policy boundaries. > > > > ??? This comes close to ignoring reality. The RFCs are less > important than what people actually send down the internet. > > What is actually sent down the Internet right now is mostly best-effort > only - the default CS0 codepoint. My inbound shaper currently shows 96GB > best-effort, 46MB CS1 and 4.3MB =E2=80=9Clow latency=E2=80=9D. > > This is called the =E2=80=9Cchicken and egg=E2=80=9D problem; application= s mostly ignore > Diffserv=E2=80=99s existence because it has no effect in most environment= s, and CPE > ignores Diffserv=E2=80=99s existence because little traffic is observed u= sing it. > > To solve the chicken-and-egg problem, you have to break that vicious > cycle. It turns out to be easier to do that on the network side, creatin= g > an environment where DSCPs *do* have effects which applications might fin= d > useful. > > > coming up with a completely different system (preferable randomized for > each home network) will make gaming the DSCPs much harder > > With all due respect, that is the single most boneheaded idea I=E2=80=99v= e come > across on this list. If the effect of applying a given DSCP is > unpredictable, and may even be opposite to the desired behaviour - or, > equivalently, if the correct DSCP to achieve a given behaviour is > unpredictable - then Diffserv will *never* be used by mainstream users an= d > applications. > > >> Cake does *not* assume that DSCPs are trustworthy. It respects them a= s > given, but employs straightforward countermeasures against misuse (eg. > higher =E2=80=9Cpriority=E2=80=9D applies only up to some fraction of cap= acity), > > > > But doesn=E2=80=99t that automatically mean that an attacker can = degrade > performance of a well configured high priority tier (with appropriate > access control) by overloading that band, which will affect the priority = of > the whole band, no? That might not be the worst alternative, but it > certainly is not side-effect free. > > If an attacker wants to cause side-effects like that, he=E2=80=99ll alway= s be able > to do so - unless he=E2=80=99s filtered at source. As a more direct coun= terpoint, > if we weren=E2=80=99t using Diffserv at all, the very same attack would d= egrade > performance for all traffic, not just the subset with equivalent DSCPs. > > Therefore, I have chosen to focus on incentivising legitimate traffic in > appropriate directions. > > >> So, if Cake gets deployed widely, an incentive for applications to > correctly mark their traffic will emerge. > > > > For which value of =E2=80=9Ccorrect=E2=80=9D exactly? > > RFC-compliant, obviously. > > There are a few very well-established DSCPs which mean =E2=80=9Cminimise = latency=E2=80=9D > (TOS4, EF) or =E2=80=9Cyield priority=E2=80=9D (CS1). The default config= uration recognises > those and treats them accordingly. > > >> But almost no program uses CS1 to label its data as lower priority > > See chicken-and-egg argument above. There are signs that CS1 is in fact > being used in its modern sense; indeed, while downloading the latest Star > Citizen test version the other day, 46MB of data ended up in CS1. Star > Citizen uses libtorrent, as I suspect do several other prominent games, s= o > adding CS1 support there would probably increase coverage quite quickly. > > >> Cake also assumes in general that the number of flows on the link at > any given instant is not too large - a few hundred is acceptable. > > > > I assume there is a build time parameter that will cater to a > specific set of flows, would recompiling with a higher value for that > constant allow to taylor cake for environments with a larger number of > concurrent flows? > > There is a compile-time constant in the code which could, in principle, b= e > exposed to the kernel configuration system. Increasing the queue count > from 1K to 32K would allow =E2=80=9Cseveral hundred=E2=80=9D to be replac= ed with =E2=80=9Cabout ten > thousand=E2=80=9D. That=E2=80=99s still not backbone-grade, but might be= useful for a very > small ISP to manage its backhaul, such as an apartment complex FTTP > installation or a village initiative. > A few years back when reading about fq_Codel and Cake, one of the research articles that I came across talked about how many flows are actually in a buffer at any given time. They looked at the buffers of backbone links from 155Mb to 10Gb and they got the same numbers every time. While these links may be servicing hundreds of thousands of active flows, at any given instant there was fewer than 200 flows in the buffer, nearly all flows had exactly one packet in the buffer, in the ballpark of 10 flows had 2 or more packets in the buffer. You could say the buffer follows the 80/20 rule. 20% of the flows in the buffer comprise of 80% of the buffer. Regardless, the total number of flows in the buffer is almost fixed. What was also interesting is the flows consuming the majority of the buffer were always in flux. You would think the same few flows that were consuming the buffer at one moment would continue to, but that is not the case, TCP keeps them alternating. When all is said and done, assuming your link is not horribly buffer-bloated, and it shouldn't be in this discussion because we're talking about fq_Codel/Cake, then there will probably be very little reason to have 32k buckets, ever. Cake especially since it has "Ways". > > - Jonathan Morton > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake > --001a11412c7a5f0497054469890b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Fri, Dec 23, 2016 at 3:53 AM, Jonathan Morton <= chromatix99@gmai= l.com> wrote:
>> As far as Diffserv is concerned, I explicitly assume that th= e standard RFC-defined DSCPs and PHBs are in use, which obviates any concer= ns about Diffserv policy boundaries.
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0??? This comes close to ignoring reality. Th= e RFCs are less important than what people actually send down the internet.=

What is actually sent down the Internet right now is mostly best-eff= ort only - the default CS0 codepoint.=C2=A0 My inbound shaper currently sho= ws 96GB best-effort, 46MB CS1 and 4.3MB =E2=80=9Clow latency=E2=80=9D.

This is called the =E2=80=9Cchicken and egg=E2=80=9D problem; applications = mostly ignore Diffserv=E2=80=99s existence because it has no effect in most= environments, and CPE ignores Diffserv=E2=80=99s existence because little = traffic is observed using it.

To solve the chicken-and-egg problem, you have to break that vicious cycle.= =C2=A0 It turns out to be easier to do that on the network side, creating a= n environment where DSCPs *do* have effects which applications might find u= seful.

> coming up with a completely different system (preferable randomized fo= r each home network) will make gaming the DSCPs much harder

With all due respect, that is the single most boneheaded idea I=E2= =80=99ve come across on this list.=C2=A0 If the effect of applying a given = DSCP is unpredictable, and may even be opposite to the desired behaviour - = or, equivalently, if the correct DSCP to achieve a given behaviour is unpre= dictable - then Diffserv will *never* be used by mainstream users and appli= cations.

>> Cake does *not* assume that DSCPs are trustworthy.=C2=A0 It respec= ts them as given, but employs straightforward countermeasures against misus= e (eg. higher =E2=80=9Cpriority=E2=80=9D applies only up to some fraction o= f capacity),
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0But doesn=E2=80=99t that automatically mean = that an attacker can degrade performance of a well configured high priority= tier (with appropriate access control) by overloading that band, which wil= l affect the priority of the whole band, no? That might not be the worst al= ternative, but it certainly is not side-effect free.

If an attacker wants to cause side-effects like that, he=E2=80=99ll = always be able to do so - unless he=E2=80=99s filtered at source.=C2=A0 As = a more direct counterpoint, if we weren=E2=80=99t using Diffserv at all, th= e very same attack would degrade performance for all traffic, not just the = subset with equivalent DSCPs.

Therefore, I have chosen to focus on incentivising legitimate traffic in ap= propriate directions.

>> So, if Cake gets deployed widely, an incentive for applications to= correctly mark their traffic will emerge.
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0For which value of =E2=80=9Ccorrect=E2=80=9D= exactly?

RFC-compliant, obviously.

There are a few very well-established DSCPs which mean =E2=80=9Cminimise la= tency=E2=80=9D (TOS4, EF) or =E2=80=9Cyield priority=E2=80=9D (CS1).=C2=A0 = The default configuration recognises those and treats them accordingly.

>> But almost no program uses CS1 to label its data as lower priority=

See chicken-and-egg argument above.=C2=A0 There are signs that CS1 i= s in fact being used in its modern sense; indeed, while downloading the lat= est Star Citizen test version the other day, 46MB of data ended up in CS1.= =C2=A0 Star Citizen uses libtorrent, as I suspect do several other prominen= t games, so adding CS1 support there would probably increase coverage quite= quickly.

>> Cake also assumes in general that the number of flows on the link = at any given instant is not too large - a few hundred is acceptable.
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0I assume there is a build time parameter tha= t will cater to a specific set of flows, would recompiling with a higher va= lue for that constant allow to taylor cake for environments with a larger n= umber of concurrent flows?

There is a compile-time constant in the code which could, in princip= le, be exposed to the kernel configuration system.=C2=A0 Increasing the que= ue count from 1K to 32K would allow =E2=80=9Cseveral hundred=E2=80=9D to be= replaced with =E2=80=9Cabout ten thousand=E2=80=9D.=C2=A0 That=E2=80=99s s= till not backbone-grade, but might be useful for a very small ISP to manage= its backhaul, such as an apartment complex FTTP installation or a village = initiative.

A few years back when readi= ng about fq_Codel and Cake, one of the research articles that I came across= talked about how many flows are actually in a buffer at any given time. Th= ey looked at the buffers of backbone links from 155Mb to 10Gb and they got = the same numbers every time. While these links may be servicing hundreds of= thousands of active flows, at any given instant there was fewer than 200 f= lows in the buffer, nearly all flows had exactly one packet in the buffer, = in the ballpark of 10 flows had 2 or more packets in the buffer.
=
You could say the buffer follows the 80/20 rule. 20% of the = flows in the buffer comprise of 80% of the buffer. Regardless, the total nu= mber of flows in the buffer is almost fixed. What was also interesting is t= he flows consuming the majority of the buffer were always in flux. You woul= d think the same few flows that were consuming the buffer at one moment wou= ld continue to, but that is not the case, TCP keeps them alternating.
=

When all is said and done, assuming your link is not ho= rribly buffer-bloated, and it shouldn't be in this discussion because w= e're talking about fq_Codel/Cake, then there will probably be very litt= le reason to have 32k buckets, ever. Cake especially since it has "Way= s".
=C2=A0

=C2=A0- Jonathan Morton

_______________________________________________
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

--001a11412c7a5f0497054469890b--