[Cake] triple flow isolation

[Cake] triple flow isolation

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

Hello List,

I've been looking at latest 'triple flow isolation' features in latest
cake git and find myself confused.  It's very likely to be a
misunderstanding on my part, although if I'm confused I'm sure others
will, sooner or later, fall into the same trap.

I thought that triple flow was a solution such that a host with many
elephant flows couldn't dominate the bandwidth consumption thus starving
another host with just one elephant flow.  I guess the typical example
is a 'bittorrent' host pulling data from many places vs a host pulling
data from a single place.  My recent 'test' example was a host doing 4
simulatenous pulls of 9GByte+ files vs my wife downloading a movie via
the TV box downstairs.  The bandwidth was evenly divided by the 5 flows,
however my host got 4/5ths of the share.  I didn't think this was
supposed to happen with triple flow isolation?  Ideally we'd both get
50% of the ISP bandwidth managed by my router (cake is running on the
WAN facing interface to the ISP modem with appropriate bandwidth limits set)

The good news is that latency was well controlled and general web
browsing/catching emails etc was oblivious to the elephant melee going on.

Advice gratefully received.


Kevin


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4816 bytes --]

Re: [Cake] triple flow isolation

[moeller0]

Hi Kevin,

I agree the triple mode seems under-documented ;)

> On Jan 11, 2016, at 18:40 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> Hello List,
> 
> I've been looking at latest 'triple flow isolation' features in latest
> cake git and find myself confused.  It's very likely to be a
> misunderstanding on my part, although if I'm confused I'm sure others
> will, sooner or later, fall into the same trap.
> 
> I thought that triple flow was a solution such that a host with many
> elephant flows couldn't dominate the bandwidth consumption thus starving
> another host with just one elephant flow.  I guess the typical example
> is a 'bittorrent' host pulling data from many places vs a host pulling
> data from a single place.  My recent 'test' example was a host doing 4
> simulatenous pulls of 9GByte+ files vs my wife downloading a movie via
> the TV box downstairs.  The bandwidth was evenly divided by the 5 flows,
> however my host got 4/5ths of the share.  I didn't think this was
> supposed to happen with triple flow isolation?  Ideally we'd both get
> 50% of the ISP bandwidth managed by my router (cake is running on the
> WAN facing interface to the ISP modem with appropriate bandwidth limits set)

	I believe you would want src_host on egress and (post-NAT) dat-host on ingress, assuming your goal is fairness by internal host, not external hosts ;). Now the challenges are to get the post-NAT IP addresses on ingress and how to counteract IPv6’s tendency to grow incredible amounts of addresses by host. In theory all of this could be solved with fairness by internal MAC addresses, except these will only work in a shallow networks (as far as I understand). I would argue that wifi aggregation has the same issue regarding IPv6, but I digress...

> 
> The good news is that latency was well controlled and general web
> browsing/catching emails etc was oblivious to the elephant melee going on.

	Well, I have not managed to get the new cake onto my router at all, so I am happy to read that it works.

Best Regards
	Sebastian

> 
> Advice gratefully received.
> 
> 
> Kevin
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] triple flow isolation

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]



On 11/01/16 18:16, moeller0 wrote:
> Hi Kevin,
>
> I agree the triple mode seems under-documented ;)
Yes that's true but it is experimental after all - and I'm experimenting
with it :-)
>> On Jan 11, 2016, at 18:40 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>>
>> Hello List,
>>
>> I've been looking at latest 'triple flow isolation' features in latest
>> cake git and find myself confused.  It's very likely to be a
>> misunderstanding on my part, although if I'm confused I'm sure others
>> will, sooner or later, fall into the same trap.
>>
>> I thought that triple flow was a solution such that a host with many
>> elephant flows couldn't dominate the bandwidth consumption thus starving
>> another host with just one elephant flow.  I guess the typical example
>> is a 'bittorrent' host pulling data from many places vs a host pulling
>> data from a single place.  My recent 'test' example was a host doing 4
>> simulatenous pulls of 9GByte+ files vs my wife downloading a movie via
>> the TV box downstairs.  The bandwidth was evenly divided by the 5 flows,
>> however my host got 4/5ths of the share.  I didn't think this was
>> supposed to happen with triple flow isolation?  Ideally we'd both get
>> 50% of the ISP bandwidth managed by my router (cake is running on the
>> WAN facing interface to the ISP modem with appropriate bandwidth limits set)
> 	I believe you would want src_host on egress and (post-NAT) dat-host on ingress, assuming your goal is fairness by internal host, not external hosts ;). Now the challenges are to get the post-NAT IP addresses on ingress and how to counteract IPv6’s tendency to grow incredible amounts of addresses by host. In theory all of this could be solved with fairness by internal MAC addresses, except these will only work in a shallow networks (as far as I understand). I would argue that wifi aggregation has the same issue regarding IPv6, but I digress...
Ah, yes, silly me - NAT.  As you say pre-NAT internal source address on
egress and post-NAT internal destination address on ingress.  So the
shaping to work, cake needs to be on the wan interface but that's too
late to obtain the internal pre-NATed addresses.  Oh dear.  Help!



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4816 bytes --]

Re: [Cake] triple flow isolation

[moeller0]

Hi Kevin,

> On Jan 11, 2016, at 21:33 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> 
> 
> On 11/01/16 18:16, moeller0 wrote:
>> Hi Kevin,
>> 
>> I agree the triple mode seems under-documented ;)
> Yes that's true but it is experimental after all - and I'm experimenting
> with it :-)
>>> On Jan 11, 2016, at 18:40 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>>> 
>>> Hello List,
>>> 
>>> I've been looking at latest 'triple flow isolation' features in latest
>>> cake git and find myself confused.  It's very likely to be a
>>> misunderstanding on my part, although if I'm confused I'm sure others
>>> will, sooner or later, fall into the same trap.
>>> 
>>> I thought that triple flow was a solution such that a host with many
>>> elephant flows couldn't dominate the bandwidth consumption thus starving
>>> another host with just one elephant flow.  I guess the typical example
>>> is a 'bittorrent' host pulling data from many places vs a host pulling
>>> data from a single place.  My recent 'test' example was a host doing 4
>>> simulatenous pulls of 9GByte+ files vs my wife downloading a movie via
>>> the TV box downstairs.  The bandwidth was evenly divided by the 5 flows,
>>> however my host got 4/5ths of the share.  I didn't think this was
>>> supposed to happen with triple flow isolation?  Ideally we'd both get
>>> 50% of the ISP bandwidth managed by my router (cake is running on the
>>> WAN facing interface to the ISP modem with appropriate bandwidth limits set)
>> 	I believe you would want src_host on egress and (post-NAT) dat-host on ingress, assuming your goal is fairness by internal host, not external hosts ;). Now the challenges are to get the post-NAT IP addresses on ingress and how to counteract IPv6’s tendency to grow incredible amounts of addresses by host. In theory all of this could be solved with fairness by internal MAC addresses, except these will only work in a shallow networks (as far as I understand). I would argue that wifi aggregation has the same issue regarding IPv6, but I digress...
> Ah, yes, silly me - NAT.  As you say pre-NAT internal source address on
> egress and post-NAT internal destination address on ingress.  So the
> shaping to work, cake needs to be on the wan interface but that's too
> late to obtain the internal pre-NATed addresses.  Oh dear.  Help!

	So a quick and dirty test would be to set up sqm with cake on the interface connecting the router SoC with its switch (aka LAN) then you have access to the internal addresses for both egress and ingress. Note that ingress and egress will be switched as compared to sqm on the wan interface so adjust the shaper settings accordingly (ingress egress are always from the view of the router, and on a LAN interface the egress direction of the interface is the ingress/download direction from the WAN). So, I would guess, destination address on egress and source address on ingress as seen by sqm should do the trick then.
	This test will obviously only work if no additional traffic hits the router besided WAN ans LAN, so no WLAN on the router for this test… Or leave everything as is an run pure IPv6 tests, since I believe openwrt does not do NAT6 by default…
	I am really curious how cake behaves in that setting...

Best Regards
	Sebastian

> 
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] triple flow isolation

[Jonathan Morton]

> On 14 Jan, 2016, at 16:20, moeller0 <moeller0@gmx.de> wrote:
> 
> I am really curious how cake behaves in that setting...

I have identified a limitation in the current triple-isolation implementation - in fact it only works properly if the sources *and* destinations of the flows are independent.  I’m working on a fix, so that it also works when *either* are different.

Turns out that a brand-new, state-of-the-art algorithm has some subtleties to it.  Who would have thought.  :-)

 - Jonathan Morton

Re: [Cake] triple flow isolation

[moeller0]

HI Jonathan,

> On Jan 14, 2016, at 15:45 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 14 Jan, 2016, at 16:20, moeller0 <moeller0@gmx.de> wrote:
>> 
>> I am really curious how cake behaves in that setting...
> 
> I have identified a limitation in the current triple-isolation implementation - in fact it only works properly if the sources *and* destinations of the flows are independent.  I’m working on a fix, so that it also works when *either* are different.
	Ah, good to know, thanks. I am still curious about the non-NAT fairness by internal IP addresses only performance, as far as I understand that is the main request/use case people seem to have. We have been offering basically per-flow fairness for some time now (and that works mostly well), but it leaves people wishing for a tool to restrict the pain under-controlled bit-torrenting can have on all users of a given home network…

> 
> Turns out that a brand-new, state-of-the-art algorithm has some subtleties to it.  Who would have thought.  :-)

;)

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] triple flow isolation

[Jonathan Morton]

> On 14 Jan, 2016, at 17:48, moeller0 <moeller0@gmx.de> wrote:
> 
> I am still curious about the non-NAT fairness by internal IP addresses only performance, as far as I understand that is the main request/use case people seem to have.

Non-NAT should work fine, once I’ve fixed the algorithm.  That’s a major part of what I intended triple-isolation to do.  It still won’t isolate a given host from it’s *own* swarm traffic, unless you also apply Diffserv, but it should prevent one host from monopolising the link with a swarm.

NAT is a problem for Cake instances operating on the “outside” of the boundary, in both directions; they see only the public IP address of the local network, and the addresses of the remote hosts.  The only real solution is probably to integrate connection tracking with the flow dissector (or to hurry along the migration to IPv6).  That’s beyond my area of expertise in the kernel.

 - Jonathan Morton

Re: [Cake] triple flow isolation

[Jonathan Morton]

> On 14 Jan, 2016, at 20:53, moeller0 <moeller0@gmx.de> wrote:
> 
> So I have not grokked the triple algorithm fully (aka not at all), but I already know that what user’s are looking for is fairness by internal host IPs. Now, since as I explained before ingress and egress really are too flexible to use as direction pointers, I assume we are looking for some configuration parameter that contains a direction; so as long as “triple” allows to request fairness by source IP or by destination IP (since these might change depending on the interface cake is running on) all will be fine. I just do not see how a simple unidirectional parameter like “triple-iso” will allow to take the fact into account that ingress and egress are only relative to the sqm interface and do not necessarily align with the internal/WAN ingress and egress… But as I said before I do not claim I understand what triple-iso intends to accomplish in detail.

The short version is that, in theory at least, I’ve found a way to ensure fairness without needing to know which side of the interface is which.  By accounting for *both* source and destination host fairness at the same time, and not placing one above the other in importance, it should all work out in the end.  The method by which I do so is probably interesting enough to write a paper about, once I’ve got it working in practice.

At this point, I strongly suspect I’ve made an implementation blunder, since even single-stepping through the theoretical algorithm’s behaviour, packet by packet, produces approximately the desired results - which are however not reproduced in actual measurements on my LAN.  Time to add more stats to the multitude already present!

If you want to be explicit about directionality, that’s what the two new “dual” modes are for.  The “triple isolation” algorithm is still used, but the undesired attribute is ignored.  The “triple” mode combines their effects.

 - Jonathan Morton

Re: [Cake] triple flow isolation

[moeller0]

Hi Jonathan,


> On Jan 15, 2016, at 01:05 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 14 Jan, 2016, at 20:53, moeller0 <moeller0@gmx.de> wrote:
>> 
>> So I have not grokked the triple algorithm fully (aka not at all), but I already know that what user’s are looking for is fairness by internal host IPs. Now, since as I explained before ingress and egress really are too flexible to use as direction pointers, I assume we are looking for some configuration parameter that contains a direction; so as long as “triple” allows to request fairness by source IP or by destination IP (since these might change depending on the interface cake is running on) all will be fine. I just do not see how a simple unidirectional parameter like “triple-iso” will allow to take the fact into account that ingress and egress are only relative to the sqm interface and do not necessarily align with the internal/WAN ingress and egress… But as I said before I do not claim I understand what triple-iso intends to accomplish in detail.
> 
> The short version is that, in theory at least, I’ve found a way to ensure fairness without needing to know which side of the interface is which.  

	Could you please explain the fairness you are talking about here? I still want to test cake in my environment and will do a much better job if I know what behavior to expect, and frankly I am unsure what triple-iso is supposed to guarantee (I know this is my own responsibility, but getting a high level description seems more efficient than trying to deduce this from the implementation, especially given that you are not fully satisfied withe the current implementation). Thanks in advance.

> By accounting for *both* source and destination host fairness at the same time, and not placing one above the other in importance, it should all work out in the end.  The method by which I do so is probably interesting enough to write a paper about, once I’ve got it working in practice.
> 
> At this point, I strongly suspect I’ve made an implementation blunder, since even single-stepping through the theoretical algorithm’s behaviour, packet by packet, produces approximately the desired results - which are however not reproduced in actual measurements on my LAN.  Time to add more stats to the multitude already present!
> 
> If you want to be explicit about directionality, that’s what the two new “dual” modes are for.  The “triple isolation” algorithm is still used, but the undesired attribute is ignored.  The “triple” mode combines their effects.

	This is the part I can not get my head around: fairness by internal host IP (what people seem to request, with, as you alluded to before, DSCP tiers per internal host IP and per flow fairness in each tier) basically will not generally allow fairness by external IP. E.g. three internal hosts talk to 6 external hosts, fairness by internal IP will try to give 1/3 of the bandwidth to the internal hosts, fairness by external IP will give 1/6, now unless the connections are symmetric (each internal IP talks to the same number of external IPs) the actual bandwidth distribution will deviate from either 1/3 or 1/6, so what level of fairness will triple-iso aim for? With explicit directionality I can easily see how to enforce either 1/3 or 1/6 in the example...

Best Regards
        Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] triple flow isolation

[Jonathan Morton]

I’ve just committed and pushed the fixes required for triple-isolation to actually work.  They are small.  My tests now pass.  This is a good feeling.

> On 15 Jan, 2016, at 10:05, moeller0 <moeller0@gmx.de> wrote:
> 
>>> I do not claim I understand what triple-iso intends to accomplish in detail.
>> 
>> The short version is that, in theory at least, I’ve found a way to ensure fairness without needing to know which side of the interface is which.  
> 
> Could you please explain the fairness you are talking about here?

As you’re already aware. Cake uses DRR++ to ensure flow-to-flow fairness.  This chiefly means keeping a deficit counter per flow, skipping over flows that are in deficit, and ensuring that the counters get incremented at a mutually equal rate, so that they eventually leave deficit.

Triple isolation also keeps such counters on a per-source-host and per-destination-host basis (NB: *not* on a per-host-pair basis).  The host deficits are checked first, and unless *both* of the relevant hosts are out of deficit, the per-flow counters are not even examined.  However, the number of flows deferred due to host deficits is tracked, which allows the host deficit counters to be batch-incremented at the correct times.  (This proved to be the main source of subtleties.)

Where two sets of flows have a common source or destination, but separate hosts at the opposite end, this ensures fair bandwidth sharing between the separate hosts, no matter which side of the link they happen to lie or the relative numbers of flows involved.  Within each such set, per-flow fairness is also ensured.  This is easy to understand and to demonstrate.

It’s also easy to see that fairness between disjoint host-pairs is also maintained in the same manner.

More complex situations require more thought to analyse.  As you suggest, the typical situation is where a small number of local hosts talk to an arbitrary combination of remote hosts through the link.  To keep matters simple, I’ll assume there are two local hosts (A and B) and that all flows are bulk in the same direction, and assert that the principles generalise to larger numbers and more realistic traffic patterns.

The simplest such case might be where A talks to a single remote host (C), but B talks to two (D and E).  This could be considered competition between a single webserver and a sharded website.  After stepping through the algorithm with some mechanical aid, I think what will happen in this case is that C-D-E fairness will govern the system, giving B more bandwidth than A.  In general, I think the side with the larger number of hosts will tend to govern.

The opposite sense would be to have the side with the smaller number of hosts govern the system.  This would, I think, handle both the swarm and shard cases better than the above, so I’ll see if I can think of a way to adapt the algorithm to do that.

 - Jonathan Morton

Re: [Cake] triple flow isolation

[Jonathan Morton]

> On 16 Jan, 2016, at 11:05, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> The opposite sense would be to have the side with the smaller number of hosts govern the system.  This would, I think, handle both the swarm and shard cases better than the above, so I’ll see if I can think of a way to adapt the algorithm to do that.

This turned out to be quite easy, going back to a slightly earlier (and happily simpler) version of the core algorithm.  I did say that the details of incrementing the per-host counters were the major source of subtleties.

https://github.com/dtaht/sch_cake/commit/2add740a79bb50b04e0497400a501df7b1857f48

 - Jonathan Morton

Re: [Cake] triple flow isolation

[moeller0]

Hi Jonathan,

> On Jan 16, 2016, at 10:35 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 16 Jan, 2016, at 11:05, Jonathan Morton <chromatix99@gmail.com> wrote:
>> 
>> The opposite sense would be to have the side with the smaller number of hosts govern the system.  This would, I think, handle both the swarm and shard cases better than the above, so I’ll see if I can think of a way to adapt the algorithm to do that.
> 
> This turned out to be quite easy, going back to a slightly earlier (and happily simpler) version of the core algorithm.  I did say that the details of incrementing the per-host counters were the major source of subtleties.
> 
> https://github.com/dtaht/sch_cake/commit/2add740a79bb50b04e0497400a501df7b1857f48

	Mmmh, I would assume that for most parts this should work well, and for the cornercases where true control seems required there is always dat_host and src_host…
Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] triple flow isolation

[moeller0]

Hi Jonathan,


> On Jan 16, 2016, at 10:05 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> I’ve just committed and pushed the fixes required for triple-isolation to actually work.  They are small.  My tests now pass.  This is a good feeling.
> 
>> On 15 Jan, 2016, at 10:05, moeller0 <moeller0@gmx.de> wrote:
>> 
>>>> I do not claim I understand what triple-iso intends to accomplish in detail.
>>> 
>>> The short version is that, in theory at least, I’ve found a way to ensure fairness without needing to know which side of the interface is which.  
>> 
>> Could you please explain the fairness you are talking about here?

	First off thanks for taking the time to explain….

> 
> As you’re already aware. Cake uses DRR++ to ensure flow-to-flow fairness.  This chiefly means keeping a deficit counter per flow, skipping over flows that are in deficit, and ensuring that the counters get incremented at a mutually equal rate, so that they eventually leave deficit.

	Okay, I am with you so far...

> 
> Triple isolation also keeps such counters on a per-source-host and per-destination-host basis (NB: *not* on a per-host-pair basis).  

	Am I right to assume that dust and src host isolation works with the same counters but simply ignores one of them? Or will only the relevant counter be kept and updated?

> The host deficits are checked first, and unless *both* of the relevant hosts are out of deficit, the per-flow counters are not even examined.  However, the number of flows deferred due to host deficits is tracked, which allows the host deficit counters to be batch-incremented at the correct times.  (This proved to be the main source of subtleties.)
> 
> Where two sets of flows have a common source or destination, but separate hosts at the opposite end, this ensures fair bandwidth sharing between the separate hosts, no matter which side of the link they happen to lie or the relative numbers of flows involved.  Within each such set, per-flow fairness is also ensured.  This is easy to understand and to demonstrate.
> 
> It’s also easy to see that fairness between disjoint host-pairs is also maintained in the same manner.
> 
> More complex situations require more thought to analyse.  As you suggest, the typical situation is where a small number of local hosts talk to an arbitrary combination of remote hosts through the link.  To keep matters simple, I’ll assume there are two local hosts (A and B) and that all flows are bulk in the same direction, and assert that the principles generalise to larger numbers and more realistic traffic patterns.
> 
> The simplest such case might be where A talks to a single remote host (C), but B talks to two (D and E).  This could be considered competition between a single webserver and a sharded website.  After stepping through the algorithm with some mechanical aid, I think what will happen in this case is that C-D-E fairness will govern the system, giving B more bandwidth than A.  In general, I think the side with the larger number of hosts will tend to govern.

	Okay.

> 
> The opposite sense would be to have the side with the smaller number of hosts govern the system.  This would, I think, handle both the swarm and shard cases better than the above, so I’ll see if I can think of a way to adapt the algorithm to do that.

	So if all internal hosts talk to one external host, does this scheme then equal pure per-flow fairness? I am trying to understand how robust triple-iso is going to be against attempt at shenanigans by unruly machines on the internal/external networks...

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] triple flow isolation

[Jonathan Morton]

> On 18 Jan, 2016, at 11:21, moeller0 <moeller0@gmx.de> wrote:
> 
> Am I right to assume that dust and src host isolation works with the same counters but simply ignores one of them?

Yes.  That’s explicit in the code.

> So if all internal hosts talk to one external host, does this scheme then equal pure per-flow fairness? I am trying to understand how robust triple-iso is going to be against attempt at shenanigans by unruly machines on the internal/external networks…

No.  If there is only one host on one side (whichever side that happens to be), then maintaining per-host fairness for that side is trivial.  The algorithm will instead maintain per-host fairness for the other side.  This derives from the fact that it also maintains per-host fairness within traffic to each individual host.  Per-flow fairness is also still maintained within individual host-pairs.

My measurements show that this aspect of the isolation is not perfect.  There is still some influence from the number of flows, which biases the actual throughput slightly from ideal per-host fairness.  This bias is however *much* smaller than pure per-flow fairness would be, and I’ve been unable to come up with a robust way of eliminating it entirely.

Hence I think it’s reasonable to simply switch on triple isolation by default, in the near future.  It does approximately the right thing, without further configuration, in the great majority of practical cases (that I can think of), and to a greater extent than the existing “flows” mode does.

I think that might also be a good time to overhaul the documentation and do some other overdue cleanup.

 - Jonathan Morton

Re: [Cake] triple flow isolation

[Alan Jenkins]

On 18/01/2016, Jonathan Morton <chromatix99@gmail.com> wrote:

> Hence I think it’s reasonable to simply switch on triple isolation by
> default, in the near future.  It does approximately the right thing, without
> further configuration, in the great majority of practical cases (that I can
> think of), and to a greater extent than the existing “flows” mode does.

A question from complete ignorance of this feature:

Do you prioritize by app ("traffic class") before imposing host
fairness, or vice versa?

We have these rules that voice, or video, can get special treatment in
order to survive (but aren't allowed to monopolize the connection).
If two local hosts simply cut the connection in half, we kind of lose
that prioritization.

E.g. when I was sharing with 4 other students - I want my house-mates
bittorrenting to stay firmly in the scavenger class when I want to use
the internet :).  (And vice-versa; I want to be able to leave my
bittorrent on and not get shouted at because netflix cuts out and
downgrades to standard definition).

Alan

Re: [Cake] triple flow isolation

[Jonathan Morton]

>> Hence I think it’s reasonable to simply switch on triple isolation by
>> default, in the near future.  It does approximately the right thing, without
>> further configuration, in the great majority of practical cases (that I can
>> think of), and to a greater extent than the existing “flows” mode does.
> 
> A question from complete ignorance of this feature:
> 
> Do you prioritize by app ("traffic class") before imposing host
> fairness, or vice versa?

It’s a fair question.  There are arguments for doing it either way.

> We have these rules that voice, or video, can get special treatment in
> order to survive (but aren't allowed to monopolize the connection).
> If two local hosts simply cut the connection in half, we kind of lose
> that prioritization.
> 
> E.g. when I was sharing with 4 other students - I want my house-mates
> bittorrenting to stay firmly in the scavenger class when I want to use
> the internet :).  (And vice-versa; I want to be able to leave my
> bittorrent on and not get shouted at because netflix cuts out and
> downgrades to standard definition).

You’ll be pleased to hear that Cake does it the way you want.  Diffserv priority queuing takes precedence over everything else bar the shaper itself.  Flow and host isolation occur within each priority tin, and AQM occurs within each flow queue.

It is entirely possible to do it the other way around.  For a wifi-specific qdisc, we might well do, in order to enforce airtime fairness between stations and then permit each station to prioritise packets within their own traffic.  This makes very different assumptions than are valid for a last-mile link.

 - Jonathan Morton

Re: [Cake] triple flow isolation

[Sebastian Moeller]

Hi Jonathan,

On January 18, 2016 10:37:35 AM GMT+01:00, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 18 Jan, 2016, at 11:21, moeller0 <moeller0@gmx.de> wrote:
>> 
>> Am I right to assume that dust and src host isolation works with the
>same counters but simply ignores one of them?
>
>Yes.  That’s explicit in the code.

        Thanks, as I am on the road since Sunday morning, I could not check the code easily.

>
>> So if all internal hosts talk to one external host, does this scheme
>then equal pure per-flow fairness? I am trying to understand how robust
>triple-iso is going to be against attempt at shenanigans by unruly
>machines on the internal/external networks…
>
>No.  If there is only one host on one side (whichever side that happens
>to be), then maintaining per-host fairness for that side is trivial. 
>The algorithm will instead maintain per-host fairness for the other
>side. This derives from the fact that it also maintains per-host
>fairness within traffic to each individual host.  Per-flow fairness is
>also still maintained within individual host-pairs.
>
>My measurements show that this aspect of the isolation is not perfect. 
>There is still some influence from the number of flows, which biases
>the actual throughput slightly from ideal per-host fairness.  This bias
>is however *much* smaller than pure per-flow fairness would be, and
>I’ve been unable to come up with a robust way of eliminating it
>entirely.

        Question: does this slight imperfection only exist for the two ,*_host settings as well? 

>
>Hence I think it’s reasonable to simply switch on triple isolation by
>default, in the near future.  It does approximately the right thing,
>without further configuration, in the great majority of practical cases
>(that I can think of), and to a greater extent than the existing
>“flows” mode does.

        Not that it matters much, but I agree, if triple-iso does work good enough and does not have any too nasty corner cases it should be the default... People requiring stricter control can always use src_ host or dst_host if they are not happy with the default I would argue...


>
>I think that might also be a good time to overhaul the documentation
>and do some other overdue cleanup.


        +1 especially for the documentation overhaul ;)

Best Regards
        Sebastian
>
> - Jonathan Morton

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.