[Bloat] tc -s qdisc questions

[Bloat] tc -s qdisc questions

[Kevin Darbyshire-Bryant]

[-- Attachment #1.1: Type: text/plain, Size: 2448 bytes --]

Hi Chaps,

Further to Dave's recent request for 'tc -s qdisc' stats looking at 
drops vs ecn marks, please could someone take a look at these and 
explain to this idiot quite what is going on?  This is a particularly 
slow & ratty ADSL line providing my parents internet 'service'.  Before 
I got hold of it with OpenWrt CC trunk + cake jammed in it, 6 seconds of 
bufferbloat was not uncommon.  I'm using the default sqm scripts that 
arrive with the 'ceropackages' repo (used as a source of patched 
iproute2, cake, etc)  In essence 2 questions: 1) Why no ECN marks, only 
drops?  2) What falls into diffserv class 0 (alias, why has nothing used 
it?)

Your time, patience etc is greatly appreciated.

Kevin

root@Router:~# tc -s qdisc show dev pppoa-wan
qdisc cake 803a: root refcnt 2 bandwidth 256Kbit diffserv4 flows
  Sent 1928303 bytes 12121 pkt (dropped 73, overlimits 6915 requeues 0)
  backlog 0b 0p requeues 0
            Class 0     Class 1     Class 2     Class 3
   rate       256Kbit     240Kbit     192Kbit      64Kbit
   target      70.5ms      75.2ms      94.0ms     282.0ms
interval     564.0ms     601.6ms     752.0ms        2.3s
Pk delay         0us      91.4ms      17.5ms       3.2ms
Av delay         0us       2.9ms       2.9ms        53us
Sp delay         0us         5us       136us         7us
   pkts             0       11754         408          32
way inds           0           0           0           0
way miss           0         347         404           5
way cols           0           0           0           0
   bytes            0     1973100       28163        6537
   drops            0          73           0           0
   marks            0           0           0           0
qdisc ingress ffff: parent ffff:fff1 ----------------
  Sent 8221609 bytes 11969 pkt (dropped 6, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
root@Router:~# tc -s qdisc show dev ifb4pppoa-wan
qdisc cake 803b: root refcnt 2 bandwidth 1512Kbit besteffort flows
  Sent 7597496 bytes 11494 pkt (dropped 482, overlimits 10487 requeues 0)
  backlog 0b 0p requeues 0
            Class 0
   rate      1512Kbit
   target      12.0ms
interval     112.0ms
Pk delay       3.3ms
Av delay       289us
Sp delay         5us
   pkts         11976
way inds           0
way miss         769
way cols           0
   bytes      8270163
   drops          482
   marks            0


[-- Attachment #1.2: Type: text/html, Size: 3707 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4791 bytes --]

Re: [Bloat] tc -s qdisc questions

[Jonathan Morton]

> On 15 May, 2015, at 01:22, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> 1) Why no ECN marks, only drops?

Probably because your hosts are not requesting ECN from servers, and therefore none of the traffic is ECN enabled.  Unfortunately, ECN requests are disabled by default in most OSes so far, but many servers will respond to them.

On Linux:
	sysctl net.ipv4.tcp_ecn=1

On Windows Vista and later:
	netsh interface tcp set global ecncapability=enabled

On MacOS X, you need two commands:
	sysctl -w net.inet.tcp.ecn_negotiate_in=1
	sysctl -w net.inet.tcp.ecn_initiate_out=1

Once set, Windows will remember it across reboots.  Linux and MacOS X require adding the appropriate entries to /etc/sysctl.conf.

> 2) What falls into diffserv class 0 (alias, why has nothing used it?

Traffic marked with the CS1 DSCP, which corresponds to “background”.  Supposedly BitTorrent is planned to do that, but I haven’t heard whether it actually does so yet.  Application support for Diffserv is generally poor so far.

 - Jonathan Morton

Re: [Bloat] tc -s qdisc questions

[Dave Taht]

On Thu, May 14, 2015 at 3:22 PM, Kevin Darbyshire-Bryant
<kevin@darbyshire-bryant.me.uk> wrote:
> Hi Chaps,
>
> Further to Dave's recent request for 'tc -s qdisc' stats looking at drops vs
> ecn marks, please could someone take a look at these and explain to this
> idiot quite what is going on?  This is a particularly slow & ratty ADSL line
> providing my parents internet 'service'.  Before I got hold of it with
> OpenWrt CC trunk + cake jammed in it, 6 seconds of bufferbloat was not
> uncommon.  I'm using the default sqm scripts that arrive with the
> 'ceropackages' repo (used as a source of patched iproute2, cake, etc)  In
> essence 2 questions: 1) Why no ECN marks, only drops?

http://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN

> 2) What falls into
> diffserv class 0 (alias, why has nothing used it?)

depends.
>
> Your time, patience etc is greatly appreciated.
>
> Kevin
>
> root@Router:~# tc -s qdisc show dev pppoa-wan
> qdisc cake 803a: root refcnt 2 bandwidth 256Kbit diffserv4 flows
>  Sent 1928303 bytes 12121 pkt (dropped 73, overlimits 6915 requeues 0)
>  backlog 0b 0p requeues 0
>            Class 0     Class 1     Class 2     Class 3
>   rate       256Kbit     240Kbit     192Kbit      64Kbit
>   target      70.5ms      75.2ms      94.0ms     282.0ms
> interval     564.0ms     601.6ms     752.0ms        2.3s
> Pk delay         0us      91.4ms      17.5ms       3.2ms
> Av delay         0us       2.9ms       2.9ms        53us
> Sp delay         0us         5us       136us         7us
>   pkts             0       11754         408          32
> way inds           0           0           0           0
> way miss           0         347         404           5
> way cols           0           0           0           0
>   bytes            0     1973100       28163        6537
>   drops            0          73           0           0
>   marks            0           0           0           0
> qdisc ingress ffff: parent ffff:fff1 ----------------
>  Sent 8221609 bytes 11969 pkt (dropped 6, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
> root@Router:~# tc -s qdisc show dev ifb4pppoa-wan
> qdisc cake 803b: root refcnt 2 bandwidth 1512Kbit besteffort flows
>  Sent 7597496 bytes 11494 pkt (dropped 482, overlimits 10487 requeues 0)
>  backlog 0b 0p requeues 0
>            Class 0
>   rate      1512Kbit
>   target      12.0ms
> interval     112.0ms
> Pk delay       3.3ms
> Av delay       289us
> Sp delay         5us
>   pkts         11976
> way inds           0
> way miss         769
> way cols           0
>   bytes      8270163
>   drops          482
>   marks            0
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] tc -s qdisc questions

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 1382 bytes --]

On 15/05/2015 00:41, Dave Taht wrote:
> On Thu, May 14, 2015 at 3:22 PM, Kevin Darbyshire-Bryant
> <kevin@darbyshire-bryant.me.uk> wrote:
>> Hi Chaps,
>>
>> Further to Dave's recent request for 'tc -s qdisc' stats looking at drops vs
>> ecn marks, please could someone take a look at these and explain to this
>> idiot quite what is going on?  This is a particularly slow & ratty ADSL line
>> providing my parents internet 'service'.  Before I got hold of it with
>> OpenWrt CC trunk + cake jammed in it, 6 seconds of bufferbloat was not
>> uncommon.  I'm using the default sqm scripts that arrive with the
>> 'ceropackages' repo (used as a source of patched iproute2, cake, etc)  In
>> essence 2 questions: 1) Why no ECN marks, only drops?
> http://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN
>
Ahh!  Penny drops, light goes on.  Thanks Jonathan & Dave.  I thought it was some magic the router performed but now I can see how silly my thinking is on that.  So presumably it would be helpful to enable ecn on the openwrt router box IF that box were producing significant traffic of its own, in addition to all the LAN side boxes?

-- 
Thanks,

Kevin@Darbyshire-Bryant.me.uk

Theresa May is watching YOU on the internet.  Join ORG
https://www.openrightsgroup.org/blog/2015/this-government-will-put-the-snoopers-charter-and-more-back-on-the-table



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4791 bytes --]

Re: [Bloat] tc -s qdisc questions

[Mikael Abrahamsson]

On Fri, 15 May 2015, Kevin Darbyshire-Bryant wrote:

> Ahh!  Penny drops, light goes on.  Thanks Jonathan & Dave.  I thought it 
> was some magic the router performed but now I can see how silly my 
> thinking is on that.  So presumably it would be helpful to enable ecn on 
> the openwrt router box IF that box were producing significant traffic of 
> its own, in addition to all the LAN side boxes?

Correct. If it for instance was running a http proxy server (and thus 
terminating TCP sessions), turning on ECN on the box itself would make 
sense.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] tc -s qdisc questions

[Dave Taht]

On Fri, May 15, 2015 at 1:55 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> On Fri, 15 May 2015, Kevin Darbyshire-Bryant wrote:
>
>> Ahh!  Penny drops, light goes on.  Thanks Jonathan & Dave.  I thought it
>> was some magic the router performed but now I can see how silly my thinking
>> is on that.  So presumably it would be helpful to enable ecn on the openwrt
>> router box IF that box were producing significant traffic of its own, in
>> addition to all the LAN side boxes?
>
>
> Correct. If it for instance was running a http proxy server (and thus
> terminating TCP sessions), turning on ECN on the box itself would make
> sense.

This was (prior to crypto on everything http starting to be mandated) a
very good idea for satellite links - run everything through a proxy, make
the proxy and proxy box be smarter about tcp/ecn - and set the fq_codel
implementation more appropriately for the rtt.

Web proxies are still quite  heavily used in education and some corporate
markets. In fact the first ever commercially deployed fq_codel implementation
(in NZ) was on a web proxy/firewall box.

I do wish more high end firewall makers were paying attention to the
bufferbloat work. They are usually the first line of defense and sole
hop before hitting the (cable,dsl,fiber)modem and thus are perfect
targets for deploying aqm/fq technologies.

Certainly the pfsense
folk are getting it but no sign of life at barracuda and checkpoint
that I know of.


> --
> Mikael Abrahamsson    email: swmike@swm.pp.se



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] tc -s qdisc questions

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

> On 15/05/2015 09:55, Mikael Abrahamsson wrote:
>> On Fri, 15 May 2015, Kevin Darbyshire-Bryant wrote:
>> 
>> Ahh!  Penny drops, light goes on.  Thanks Jonathan & Dave.  I thought it was some magic the router performed but now I can see how silly my thinking is on that.  So presumably it would be helpful to enable ecn on the openwrt router box IF that box were producing significant traffic of its own, in addition to all the LAN side boxes?
> 
> Correct. If it for instance was running a http proxy server (and thus terminating TCP sessions), turning on ECN on the box itself would make sense.

Thanks Mikael - some ECN tweakage in progress.  I'll see which bits of the Internet I use break :-)

Kevin


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3089 bytes --]

Re: [Bloat] tc -s qdisc questions

[Mikael Abrahamsson]

On Sun, 17 May 2015, Kevin Darbyshire-Bryant wrote:

> Thanks Mikael - some ECN tweakage in progress.  I'll see which bits of 
> the Internet I use break :-)

I have been running this on my macbook for almost 2 years (since I got 
it):

$ cat /etc/sysctl.conf
net.inet.tcp.ecn_initiate_out=1
net.inet.tcp.ecn_negotiate_in=1

and this on my linux server for 5+ years:

$ cat /etc/sysctl.conf | grep -i ecn
net.ipv4.tcp_ecn = 1

I used the same on my previous Ubuntu based laptop for 5+ years.

I have yet to encounter any ECN related breakage. I have had significant 
problems with IPv6 PMTUD blackholing, but no ECN related problems. I had 
ECN related problems back in 2001 (or whenever it was)  when Linux first 
introduced it due to a lot of firewalls being very paranoid about it, but 
then I turned it back on again 5-7 years ago and by then all problems were 
gone.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] tc -s qdisc questions

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]



On 18/05/15 06:46, Mikael Abrahamsson wrote:
> On Sun, 17 May 2015, Kevin Darbyshire-Bryant wrote:
>
>> Thanks Mikael - some ECN tweakage in progress.  I'll see which bits 
>> of the Internet I use break :-)
>
> I have been running this on my macbook for almost 2 years (since I got 
> it):
>
> $ cat /etc/sysctl.conf
> net.inet.tcp.ecn_initiate_out=1
> net.inet.tcp.ecn_negotiate_in=1
>
> and this on my linux server for 5+ years:
>
> $ cat /etc/sysctl.conf | grep -i ecn
> net.ipv4.tcp_ecn = 1
>
> I used the same on my previous Ubuntu based laptop for 5+ years.
>
> I have yet to encounter any ECN related breakage. I have had 
> significant problems with IPv6 PMTUD blackholing, but no ECN related 
> problems. I had ECN related problems back in 2001 (or whenever it 
> was)  when Linux first introduced it due to a lot of firewalls being 
> very paranoid about it, but then I turned it back on again 5-7 years 
> ago and by then all problems were gone.
>
That's good to know :-)

And for those struggling (ok just me then) to get ecn to survive a 
reboot on OpenWrt despite playing with sysctl.conf, instead add "option 
tcp_ecn '1'" in /etc/config/firewall something like:


config defaults
         option syn_flood '1'
         option input 'ACCEPT'
         option output 'ACCEPT'
         option forward 'REJECT'
         option drop_invalid '1'
         option tcp_ecn '1'


Kevin (being that 'better built idiot' since 1971) :-)





[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4791 bytes --]