From: Sebastian Moeller <moeller0@gmx.de>
To: Georgios Amanakis <g_amanakis@yahoo.com>
Cc: cake@lists.bufferbloat.net
Subject: Re: [Cake] WAN ingress rate with concurrent downloads
Date: Sun, 30 Oct 2016 20:27:27 +0100 [thread overview]
Message-ID: <0D26D833-A8E0-488D-840C-2B63E853C555@gmx.de> (raw)
In-Reply-To: <1477845791.1404.2.camel@yahoo.com>
Hi Georgios,
> On Oct 30, 2016, at 17:43, Georgios Amanakis <g_amanakis@yahoo.com> wrote:
>
> Dear All,
>
> I think Sebastian's theory has a weak point.
Oh, I do not claim to be an expert on these matters, so would cherish input from others, but I do think my hypothesis is not to be dismissed completely (see below).
>
>> Again I believe you just show bit-torrent misbehaving ...
>
> I can easily reproduce this behaviour if one LAN-host downloads
> multiple files using TCP (e.g. 2 or more kernels simultaneously from ht
> tp://www.kernel.org).
>
>> For me this looks like bwmon accounts the incoming packets before
>> cake sees and drops them ...
>
> If only one download takes place then the bandwidth bmon reports on
> WAN-ingress is the same as the limit set in cake. I don't think that
> bmon accounts the incoming packets before cake sees them.
That could be true, but also should be testable. According to http://bwmon.sourceforge.net bwmon reads /proc/net/dev to get its estimates, so “cat /proc/net/dev” should show you what is in there:
Directly after reboot:
root@turris:~# cat /proc/net/dev
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
gre0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan0: 526875 2287 0 0 0 0 0 0 693742 2001 0 0 0 0 0 0
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
gretap0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 2175 26 0 0 0 0 0 0 2175 26 0 0 0 0 0 0
ifb4eth1: 740189 2097 0 0 0 0 0 0 740189 2097 0 0 0 0 0 0
eth0: 0 0 0 0 0 0 0 0 17543 84 0 0 0 0 0 0
eth1: 752467 2167 0 0 0 0 0 0 541001 2390 0 0 0 0 0 0
eth2: 0 0 0 0 0 0 0 0 17497 83 0 0 0 0 0 0
ip6tnl0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan1: 0 0 0 0 0 0 0 0 17875 76 0 0 0 0 0 0
br-lan: 494863 2288 0 0 0 0 0 0 627972 1881 0 0 0 0 0 0
ifb0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
teql0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ifb1: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
After a dslreports speedtest with 16/16 streams:
root@turris:~# cat /proc/net/dev
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
gre0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan0: 20160809 37095 0 0 0 0 0 0 48570975 44819 0 0 0 0 0 0
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
gretap0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 2453 28 0 0 0 0 0 0 2453 28 0 0 0 0 0 0
ifb4eth1: 47764269 44632 0 0 0 0 0 0 47764269 44632 0 0 0 0 0 0
eth0: 0 0 0 0 0 0 0 0 20435 100 0 0 0 0 0 0
eth1: 48135827 45333 0 0 0 0 0 0 19357920 36538 0 0 0 0 0 0
eth2: 0 0 0 0 0 0 0 0 20389 99 0 0 0 0 0 0
ip6tnl0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan1: 0 0 0 0 0 0 0 0 21047 92 0 0 0 0 0 0
br-lan: 19641491 37097 0 0 0 0 0 0 47617521 44254 0 0 0 0 0 0
ifb0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
teql0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ifb1: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
root@turris:~#
Please notice how eth1 reports initially 2167-2097 = 70 packets more than ifb4eth1 (the shapers ingress qdisc is attached to the IFB as linux does not allow shapers on ingress without the ifb trick, policers however can directly act in ingress). After the test we see 45333-44632 = 701 Packets more on eth1 than ifb4eth1. I take this as supporting evidence for my hypothesis...
> Furthermore,
> Wireshark reports that all bittorrent traffic in my case are TCP
> packets, and reports the same bandwidth usage as bmon on WAN-ingress.
Well, if both look at the raw enp0s13 interface I would expect both of them to report the traffic before packets can be dropped.
>
> The only way I have found that WAN-ingress limit is obeyed, is using
> act_police with the in-kernel bandwidth estimator (i.e. tc filter add
> dev enp0s13 parent ffff: protocol all estimator 10msec 80msec u32 match
> u32 0 0 police avrate 3300kbit drop flowid :1).
As stated above, I believe I recall that policers can directly act on ingress and hence affect the number of packets reported for that interface...
> But then there is an
> discrepancy between WAN-ingress and LAN-egress (i.e. WAN-ingress is at
> ~400kbytes/s and LAN-egress on the host running bittorrent is
> ~220kbytes/s). Unfortunately I cannot see the bandwidth my cable-modem
> syncs at (I searched the web-interface but did not find it; it is a
> Comcast-provided Arris TG1682G).
With DOCSIS systems there is no clear and easy way to get hod on the relevant numbers (which would be the numbers your ISP plugged into the CMTS shaper and information about potentially allowed bursts).
>
> Noah, could you try and see what happens if instead of bittorrent a
> LAN-host downloads multiple files using TCP?
I would also be interested in the outcome of that test, please keep posting to the list.
Best Regards
Sebastian
>
> Thank you all for your feedback,
> George
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
next prev parent reply other threads:[~2016-10-30 19:27 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <mailman.1.1477670401.20617.cake@lists.bufferbloat.net>
2016-10-30 16:43 ` Georgios Amanakis
2016-10-30 19:27 ` Sebastian Moeller [this message]
[not found] <1462124589.1190968.1481310707746.ref@mail.yahoo.com>
2016-12-09 19:11 ` George Amanakis
2016-12-10 19:33 ` Benjamin Cronce
[not found] <mailman.5.1477497601.28053.cake@lists.bufferbloat.net>
2016-10-27 0:10 ` G. Amanakis
2016-10-27 0:16 ` G. Amanakis
2016-10-27 6:41 ` Sebastian Moeller
2016-10-27 16:42 ` Noah Causin
2016-10-28 1:34 ` Georgios Amanakis
2016-10-28 7:45 ` Sebastian Moeller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://lists.bufferbloat.net/postorius/lists/cake.lists.bufferbloat.net/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0D26D833-A8E0-488D-840C-2B63E853C555@gmx.de \
--to=moeller0@gmx.de \
--cc=cake@lists.bufferbloat.net \
--cc=g_amanakis@yahoo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox