Re: [Cake] bufferbloat still misunderstood & ignored

Re: [Cake] bufferbloat still misunderstood & ignored

[Toke Høiland-Jørgensen]

Kevin Darbyshire-Bryant via Cake <cake@lists.bufferbloat.net> writes:

> From: Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk>
> Subject: bufferbloat still misunderstood & ignored
> To: Cake List <cake@lists.bufferbloat.net>
> Date: Wed, 28 Mar 2018 15:46:47 +0000
>
> http://forums.thinkbroadband.com/talktalk/t/4587923-sensible-discussion-with-talktalk-about-bufferbloat.html
>
> The thing that bothers me more than anything….. the first reply comes
> from a staff member of ‘thinkbroadband’.

"Solution is two get two lines, so if a big download is using up one
they can do latency sensitive stuff on the other."

Lovely; an ISP that found a way to monetise bufferbloat... :/

-Toke

Re: [Cake] bufferbloat still misunderstood & ignored

[Dave Taht]

The two line method is quite common among gamers.

Re: [Cake] bufferbloat still misunderstood & ignored

[Jonathan Morton]

> On 28 Mar, 2018, at 10:32 pm, Dave Taht <dave.taht@gmail.com> wrote:
> 
> The two line method is quite common among gamers.

I'm pretty sure a single A&A ADSL connection costs less than two from a bargain-basement ISP, what with line rental factoring into the total price.  Of course, not everyone is lucky enough to live in the UK and thus have access to A&A (myself included).

Something else to shoehorn into the Cake paper: as future work, an ISP-focused qdisc (or even a whole middlebox using commodity hardware) using a rearranged version of the technology proven in Cake.  Cake itself is focused on managing a single link, whereas an ISP must deal with many subscriber links (with a finite variety of service tiers) sharing a backhaul link, with the link interface hardware itself being unable to avoid bufferbloat or to share bandwidth fairly when congestion occurs.  The backhaul link might be to a local exchange, a shared cable medium, or a satellite hop.

 - Jonathan Morton

Re: [Cake] bufferbloat still misunderstood & ignored

[Dave Taht]

On Wed, Mar 28, 2018 at 5:04 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>> On 28 Mar, 2018, at 10:32 pm, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> The two line method is quite common among gamers.
>
> I'm pretty sure a single A&A ADSL connection costs less than two from a bargain-basement ISP, what with line rental factoring into the total price.  Of course, not everyone is lucky enough to live in the UK and thus have access to A&A (myself included).
>
> Something else to shoehorn into the Cake paper: as future work, an ISP-focused qdisc (or even a whole middlebox using commodity hardware) using a rearranged version of the technology proven in Cake.  Cake itself is focused on managing a single link, whereas an ISP must deal with many subscriber links (with a finite variety of service tiers) sharing a backhaul link, with the link interface hardware itself being unable to avoid bufferbloat or to share bandwidth fairly when congestion occurs.  The backhaul link might be to a local exchange, a shared cable medium, or a satellite hop.

Everything can be solved with one more level of indirection. In this
case a full solution would be to be able to cleanly pattern match
against nexthop mac, or the set of ipv4 and ipv6 addresses the
subscriber has, and feed each unique match into a queue that feeds
into a cake-like instance. A cake of cakes.

A partial solution could be one virtual interface per subscriber.

A finicky bit would be who to penalize when the underlying medium
(shared cable) is oversubscribed.

>
>  - Jonathan Morton



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] bufferbloat still misunderstood & ignored

[Jonathan Morton]

> On 29 Mar, 2018, at 3:26 am, Dave Taht <dave.taht@gmail.com> wrote:
> 
> A finicky bit would be who to penalize when the underlying medium
> (shared cable) is oversubscribed.

Two obvious reasonable solutions: share equally per subscriber, or share proportionately to provisioned bandwidth per subscriber.  Either should be fairly straightforward to implement in an integrated qdisc, and either would penalise the (instantaneously) heaviest users before affecting normal or light users.

Equal sharing has the interesting side-effect that subscribers on lower tiers don't notice backhaul congestion at all until higher tiers have been forced down to their level.  This potentially gives ISPs an incentive to avoid such extreme congestion (by upgrading backhaul to match demand), since rational customers won't pay for bandwidth they can't use.  It also ensures that all subscribers retain a reasonable, basic level of service during abnormal congestion events.

Conversely, proportional sharing might give a perverse incentive, since paying more gives a larger share of the pie, no matter how cramped it is.  Artificial scarcity could then be used to aid up-selling in an anti-consumer manner, similar to what's been seen with Netflix.  It would be naive to assume that ISPs won't do this, given the opportunity, so it would be better to build only the more consumer-friendly option into the software.

Theoretically, a middle ground could be to assign a sharing weight separately from the provisioned bandwidth.  This would permit, for example, subscribers provisioned at 100:1 bandwidths to receive 4:1 service under congested conditions.  However, this would be under ISPs' control and fully documented, and would therefore be a little too tempting to abuse.

 - Jonathan Morton

Re: [Cake] bufferbloat still misunderstood & ignored

[Dave Taht]

I so wish that the network nuetrality debate included discussions such as these.

On Wed, Mar 28, 2018 at 5:53 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>> On 29 Mar, 2018, at 3:26 am, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> A finicky bit would be who to penalize when the underlying medium
>> (shared cable) is oversubscribed.
>
> Two obvious reasonable solutions: share equally per subscriber, or share proportionately to provisioned bandwidth per subscriber.  Either should be fairly straightforward to implement in an integrated qdisc, and either would penalise the (instantaneously) heaviest users before affecting normal or light users.
>
> Equal sharing has the interesting side-effect that subscribers on lower tiers don't notice backhaul congestion at all until higher tiers have been forced down to their level.  This potentially gives ISPs an incentive to avoid such extreme congestion (by upgrading backhaul to match demand), since rational customers won't pay for bandwidth they can't use.  It also ensures that all subscribers retain a reasonable, basic level of service during abnormal congestion events.
>
> Conversely, proportional sharing might give a perverse incentive, since paying more gives a larger share of the pie, no matter how cramped it is.  Artificial scarcity could then be used to aid up-selling in an anti-consumer manner, similar to what's been seen with Netflix.  It would be naive to assume that ISPs won't do this, given the opportunity, so it would be better to build only the more consumer-friendly option into the software.
>
> Theoretically, a middle ground could be to assign a sharing weight separately from the provisioned bandwidth.  This would permit, for example, subscribers provisioned at 100:1 bandwidths to receive 4:1 service under congested conditions.  However, this would be under ISPs' control and fully documented, and would therefore be a little too tempting to abuse.
>
>  - Jonathan Morton
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] bufferbloat still misunderstood & ignored

[Andy Furniss]

Ironically, in the UK my cheap ISP, Plusnet used to do QOS for free.
The ASA (Advertising standards authority) decreed that ISPs that mark 
traffic can't claim "totally unlimited" in ads - so they turned it off.
You can now pay more to opt into something similar.
It could be of course that there is more to it - eg. excuse to save on 
kit for marking, or the ASA considered that internal "discrimination" 
was going on - but IME over years with them I never saw evidence of that.
Anyway for a single line I think Plusnet and opt in/pay more would be 
cheaper than AA (who IIRC, historically at least, don't classify/mark 
but just do something simple like prio smaller packets - though I 
haven't checked what they do now).

Dave Taht wrote:
> I so wish that the network nuetrality debate included discussions such as these.
> 
> On Wed, Mar 28, 2018 at 5:53 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>>> On 29 Mar, 2018, at 3:26 am, Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> A finicky bit would be who to penalize when the underlying medium
>>> (shared cable) is oversubscribed.
>>
>> Two obvious reasonable solutions: share equally per subscriber, or share proportionately to provisioned bandwidth per subscriber.  Either should be fairly straightforward to implement in an integrated qdisc, and either would penalise the (instantaneously) heaviest users before affecting normal or light users.
>>
>> Equal sharing has the interesting side-effect that subscribers on lower tiers don't notice backhaul congestion at all until higher tiers have been forced down to their level.  This potentially gives ISPs an incentive to avoid such extreme congestion (by upgrading backhaul to match demand), since rational customers won't pay for bandwidth they can't use.  It also ensures that all subscribers retain a reasonable, basic level of service during abnormal congestion events.
>>
>> Conversely, proportional sharing might give a perverse incentive, since paying more gives a larger share of the pie, no matter how cramped it is.  Artificial scarcity could then be used to aid up-selling in an anti-consumer manner, similar to what's been seen with Netflix.  It would be naive to assume that ISPs won't do this, given the opportunity, so it would be better to build only the more consumer-friendly option into the software.
>>
>> Theoretically, a middle ground could be to assign a sharing weight separately from the provisioned bandwidth.  This would permit, for example, subscribers provisioned at 100:1 bandwidths to receive 4:1 service under congested conditions.  However, this would be under ISPs' control and fully documented, and would therefore be a little too tempting to abuse.
>>
>>   - Jonathan Morton
>>
> 
> 
> 

Re: [Cake] bufferbloat still misunderstood & ignored

[Pete Heist]

> On Mar 29, 2018, at 2:53 AM, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> On 29 Mar, 2018, at 3:26 am, Dave Taht <dave.taht@gmail.com> wrote:
>> 
>> A finicky bit would be who to penalize when the underlying medium
>> (shared cable) is oversubscribed.
> 
> Two obvious reasonable solutions: share equally per subscriber, or share proportionately to provisioned bandwidth per subscriber.  Either should be fairly straightforward to implement in an integrated qdisc, and either would penalise the (instantaneously) heaviest users before affecting normal or light users.

As one use case, FreeNet Liberec, the co-operative ISP I use, would use the “shared equally” option. The important part for them is that some members access from a single CPE device and others from multiple devices whose IPs are handed out by FreeNet’s routers. There’s a db with mappings from member ID to their MACs, I believe, which is used to generate various config files. So this mapping from member (subscriber) to their MACs or IPs would need to be configurable somewhere, which was already eluded to earlier.

Personally, I like the idea of _only_ offering equal sharing, as it’s easiest to implement and serves a social purpose.

Re: [Cake] bufferbloat still misunderstood & ignored

[Jonathan Morton]

> On 30 Mar, 2018, at 11:05 am, Pete Heist <pete@eventide.io> wrote:
> 
> So this mapping from member (subscriber) to their MACs or IPs would need to be configurable somewhere

Yes, I assumed something like that would be required to assign the correct tier of service (or BRAS rate) to each subscriber, and that's what makes it ISP-focused rather than end-user focused.

In Linux it appears to be possible to assign a flow mark (ie. a number) to packets using firewall rules or qdisc filters, and the subscriber ID and tier ID could probably be encoded into that.  It would then only be necessary to inform the qdisc of this encoding (how many bits for each portion) and to define the tiers.

It would also probably be more appropriate to give an ISP-focused qdisc a sysfs configuration/stats interface with more flexibility than an iproute2 interface.  I still think Cake works fine with iproute2 alone.

 - Jonathan Morton

Re: [Cake] bufferbloat still misunderstood & ignored

[Toke Høiland-Jørgensen]

Jonathan Morton <chromatix99@gmail.com> writes:

>> On 30 Mar, 2018, at 11:05 am, Pete Heist <pete@eventide.io> wrote:
>> 
>> So this mapping from member (subscriber) to their MACs or IPs would need to be configurable somewhere
>
> Yes, I assumed something like that would be required to assign the
> correct tier of service (or BRAS rate) to each subscriber, and that's
> what makes it ISP-focused rather than end-user focused.
>
> In Linux it appears to be possible to assign a flow mark (ie. a
> number) to packets using firewall rules or qdisc filters, and the
> subscriber ID and tier ID could probably be encoded into that. It
> would then only be necessary to inform the qdisc of this encoding (how
> many bits for each portion) and to define the tiers.

Even better, you can do arbitrary programmable matching these days,
using tc-bpf(8). Theoretically FQ-CoDel supports this in place of the
built-in flow hashing; but I have never managed to get it configured
correctly (haven't tried that hard, though).

-Toke