[Cake] Cake strange behaviour

[Cake] Cake strange behaviour

[Kevin Darbyshire-Bryant]

Hi guys,

Encountering some behaviour that I don't understand.  Line is a 40/10 
cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in ingress, 
'dual-srcshosts' on engress - limiting the on the WAN line. Take a look 
at my ping response graph

http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html

Around 20:30 I fired up a windows machine that was behind on its updates 
so it generated a bit of ingress traffic.  Note the comparatively high 
latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms 
steady (blue) latency you can see is a system backup (so egress traffic) 
running till around 23:00.

The really strange bit is that cake stats show it has only dropped 10 
(yes 10!) packets.

I'm not the only person encountering 'interesting' behaviour with regard 
to windows updates inducing high latency and high packet loss.  It's as 
if cake weren't there managing flows and this is the ISP's rate limiter 
in action.

Kevin

Re: [Cake] Cake strange behaviour

[Dave Täht]

I would repeat the same test with htb+fq_codel.

On 7/16/16 11:35 AM, Kevin Darbyshire-Bryant wrote:
> Hi guys,
> 
> Encountering some behaviour that I don't understand.  Line is a 40/10
> cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in ingress,
> 'dual-srcshosts' on engress - limiting the on the WAN line. Take a look
> at my ping response graph
> 
> http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html
> 
> 
> Around 20:30 I fired up a windows machine that was behind on its updates
> so it generated a bit of ingress traffic.  Note the comparatively high
> latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms
> steady (blue) latency you can see is a system backup (so egress traffic)
> running till around 23:00.
> 
> The really strange bit is that cake stats show it has only dropped 10
> (yes 10!) packets.
> 
> I'm not the only person encountering 'interesting' behaviour with regard
> to windows updates inducing high latency and high packet loss.  It's as
> if cake weren't there managing flows and this is the ISP's rate limiter
> in action.
> 
> Kevin
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Cake strange behaviour

[Kevin Darbyshire-Bryant]

On 16/07/16 11:59, Dave Täht wrote:
> I would repeat the same test with htb+fq_codel.

Hi Dave,

That's more challenging than it sounds - reproducing the test scenario 
would require the windows box going back in time.  What could it be 
doing that so far any of the flent tests fail to replicate?  Hmmm, so 
far I've used a local flent server...I wonder if RTT is at play here?

Kevin

>
> On 7/16/16 11:35 AM, Kevin Darbyshire-Bryant wrote:
>> Hi guys,
>>
>> Encountering some behaviour that I don't understand.  Line is a 40/10
>> cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in ingress,
>> 'dual-srcshosts' on engress - limiting the on the WAN line. Take a look
>> at my ping response graph
>>
>> http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html
>>
>>
>> Around 20:30 I fired up a windows machine that was behind on its updates
>> so it generated a bit of ingress traffic.  Note the comparatively high
>> latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms
>> steady (blue) latency you can see is a system backup (so egress traffic)
>> running till around 23:00.
>>
>> The really strange bit is that cake stats show it has only dropped 10
>> (yes 10!) packets.
>>
>> I'm not the only person encountering 'interesting' behaviour with regard
>> to windows updates inducing high latency and high packet loss.  It's as
>> if cake weren't there managing flows and this is the ISP's rate limiter
>> in action.
>>
>> Kevin
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Cake strange behaviour

[Dave Täht]

On 7/16/16 1:53 PM, Kevin Darbyshire-Bryant wrote:
> 
> 
> On 16/07/16 11:59, Dave Täht wrote:
>> I would repeat the same test with htb+fq_codel.
> 
> Hi Dave,
> 
> That's more challenging than it sounds - reproducing the test scenario
> would require the windows box going back in time.  What could it be
> doing that so far any of the flent tests fail to replicate?  Hmmm, so
> far I've used a local flent server...I wonder if RTT is at play here?

Without extensive testing I have no faith that the current aqm
implementation in cake is actually working. In the case of inbound
traffic, managed at the cpe rather than at the isp, fq alone does not
manage to reduce queue length.

As for the actual characeristics of microsofts new distribution system,
I have heard many reports of it slamming networks, but have no traces
to work from. A suggestion to get some would be to put a freshly
installed windows box on the network and capture all that happens.

> Kevin
> 
>>
>> On 7/16/16 11:35 AM, Kevin Darbyshire-Bryant wrote:
>>> Hi guys,
>>>
>>> Encountering some behaviour that I don't understand.  Line is a 40/10
>>> cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in ingress,
>>> 'dual-srcshosts' on engress - limiting the on the WAN line. Take a look
>>> at my ping response graph
>>>
>>> http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html
>>>
>>>
>>>
>>> Around 20:30 I fired up a windows machine that was behind on its updates
>>> so it generated a bit of ingress traffic.  Note the comparatively high
>>> latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms
>>> steady (blue) latency you can see is a system backup (so egress traffic)
>>> running till around 23:00.
>>>
>>> The really strange bit is that cake stats show it has only dropped 10
>>> (yes 10!) packets.
>>>
>>> I'm not the only person encountering 'interesting' behaviour with regard
>>> to windows updates inducing high latency and high packet loss.  It's as
>>> if cake weren't there managing flows and this is the ISP's rate limiter
>>> in action.
>>>
>>> Kevin
>>>
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Cake strange behaviour

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 2391 bytes --]

Hi Kevin,

Silly idea, use the Windows rollback point (or what is it called) from before the update and then run Windows upgrade again.

Sebastian

On July 16, 2016 1:53:20 PM GMT+02:00, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>
>
>On 16/07/16 11:59, Dave Täht wrote:
>> I would repeat the same test with htb+fq_codel.
>
>Hi Dave,
>
>That's more challenging than it sounds - reproducing the test scenario 
>would require the windows box going back in time.  What could it be 
>doing that so far any of the flent tests fail to replicate?  Hmmm, so 
>far I've used a local flent server...I wonder if RTT is at play here?
>
>Kevin
>
>>
>> On 7/16/16 11:35 AM, Kevin Darbyshire-Bryant wrote:
>>> Hi guys,
>>>
>>> Encountering some behaviour that I don't understand.  Line is a
>40/10
>>> cake limited to 39000/9840. Overheads 12, 'dual-dsthosts' in
>ingress,
>>> 'dual-srcshosts' on engress - limiting the on the WAN line. Take a
>look
>>> at my ping response graph
>>>
>>>
>http://www.thinkbroadband.com/ping/share/9822cb5160582fa6abee29b60d807766-16-07-2016.html
>>>
>>>
>>> Around 20:30 I fired up a windows machine that was behind on its
>updates
>>> so it generated a bit of ingress traffic.  Note the comparatively
>high
>>> latency (40ms) and stupidly high ping packet loss (50%)  The 3-5ms
>>> steady (blue) latency you can see is a system backup (so egress
>traffic)
>>> running till around 23:00.
>>>
>>> The really strange bit is that cake stats show it has only dropped
>10
>>> (yes 10!) packets.
>>>
>>> I'm not the only person encountering 'interesting' behaviour with
>regard
>>> to windows updates inducing high latency and high packet loss.  It's
>as
>>> if cake weren't there managing flows and this is the ISP's rate
>limiter
>>> in action.
>>>
>>> Kevin
>>>
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>
>_______________________________________________
>Cake mailing list
>Cake@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/cake

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 3285 bytes --]

Re: [Cake] Cake strange behaviour

[Jonathan Morton]

> Line is a 40/10 cake limited to 39000/9840.

I think you might be rather optimistic on your ingress shaping rate.  In order to control the queue effectively in the (less ideal) downstream position, the shaper needs to be set somewhat below the actual line rate.  I’m not yet sure exactly how far down it needs to be, but just 2.5% is probably not enough, especially in the face of a traffic type reputed to be “intensive".

> Without extensive testing I have no faith that the current aqm implementation in cake is actually working.

Meanwhile, I’ve been downloading a fresh copy of Star Citizen for the past 12+ hours, using the extremely intensive P2P method.  Cake reported 300-500 *bulk* flows at various times, all in the Best Effort tin.  Web browsing on the same machine is predictably slow, but actually works (aside from servers which automatically cut connections after a timeout, even if they aren’t idle).

But web browsing on a *different* machine is still responsive, even with fairly large images, thanks to triple-isolation.  That wouldn’t happen unless the upstream queue was being kept empty - a very difficult task under that sort of load.

All this is greatly helped by the fact that my 3G ISP has apparently just upgraded a nearby tower, so I am now able to set 4Mbps ingress without having to account for a lot of variability.  Previously I had to set 1Mbps ingress to be sure of low latencies.

I really should get on with migrating my firewall to a different box, to free up the one with two NICs.  Then I can run the sort of tests that might actually convince people.

 - Jonathan Morton