[Cake] WAN ingress rate with concurrent downloads
Sebastian Moeller
moeller0 at gmx.de
Sun Oct 30 15:27:27 EDT 2016
Hi Georgios,
> On Oct 30, 2016, at 17:43, Georgios Amanakis <g_amanakis at yahoo.com> wrote:
>
> Dear All,
>
> I think Sebastian's theory has a weak point.
Oh, I do not claim to be an expert on these matters, so would cherish input from others, but I do think my hypothesis is not to be dismissed completely (see below).
>
>> Again I believe you just show bit-torrent misbehaving ...
>
> I can easily reproduce this behaviour if one LAN-host downloads
> multiple files using TCP (e.g. 2 or more kernels simultaneously from ht
> tp://www.kernel.org).
>
>> For me this looks like bwmon accounts the incoming packets before
>> cake sees and drops them ...
>
> If only one download takes place then the bandwidth bmon reports on
> WAN-ingress is the same as the limit set in cake. I don't think that
> bmon accounts the incoming packets before cake sees them.
That could be true, but also should be testable. According to http://bwmon.sourceforge.net bwmon reads /proc/net/dev to get its estimates, so “cat /proc/net/dev” should show you what is in there:
Directly after reboot:
root at turris:~# cat /proc/net/dev
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
gre0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan0: 526875 2287 0 0 0 0 0 0 693742 2001 0 0 0 0 0 0
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
gretap0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 2175 26 0 0 0 0 0 0 2175 26 0 0 0 0 0 0
ifb4eth1: 740189 2097 0 0 0 0 0 0 740189 2097 0 0 0 0 0 0
eth0: 0 0 0 0 0 0 0 0 17543 84 0 0 0 0 0 0
eth1: 752467 2167 0 0 0 0 0 0 541001 2390 0 0 0 0 0 0
eth2: 0 0 0 0 0 0 0 0 17497 83 0 0 0 0 0 0
ip6tnl0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan1: 0 0 0 0 0 0 0 0 17875 76 0 0 0 0 0 0
br-lan: 494863 2288 0 0 0 0 0 0 627972 1881 0 0 0 0 0 0
ifb0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
teql0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ifb1: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
After a dslreports speedtest with 16/16 streams:
root at turris:~# cat /proc/net/dev
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
gre0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan0: 20160809 37095 0 0 0 0 0 0 48570975 44819 0 0 0 0 0 0
sit0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
gretap0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
lo: 2453 28 0 0 0 0 0 0 2453 28 0 0 0 0 0 0
ifb4eth1: 47764269 44632 0 0 0 0 0 0 47764269 44632 0 0 0 0 0 0
eth0: 0 0 0 0 0 0 0 0 20435 100 0 0 0 0 0 0
eth1: 48135827 45333 0 0 0 0 0 0 19357920 36538 0 0 0 0 0 0
eth2: 0 0 0 0 0 0 0 0 20389 99 0 0 0 0 0 0
ip6tnl0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
wlan1: 0 0 0 0 0 0 0 0 21047 92 0 0 0 0 0 0
br-lan: 19641491 37097 0 0 0 0 0 0 47617521 44254 0 0 0 0 0 0
ifb0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
teql0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ifb1: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
root at turris:~#
Please notice how eth1 reports initially 2167-2097 = 70 packets more than ifb4eth1 (the shapers ingress qdisc is attached to the IFB as linux does not allow shapers on ingress without the ifb trick, policers however can directly act in ingress). After the test we see 45333-44632 = 701 Packets more on eth1 than ifb4eth1. I take this as supporting evidence for my hypothesis...
> Furthermore,
> Wireshark reports that all bittorrent traffic in my case are TCP
> packets, and reports the same bandwidth usage as bmon on WAN-ingress.
Well, if both look at the raw enp0s13 interface I would expect both of them to report the traffic before packets can be dropped.
>
> The only way I have found that WAN-ingress limit is obeyed, is using
> act_police with the in-kernel bandwidth estimator (i.e. tc filter add
> dev enp0s13 parent ffff: protocol all estimator 10msec 80msec u32 match
> u32 0 0 police avrate 3300kbit drop flowid :1).
As stated above, I believe I recall that policers can directly act on ingress and hence affect the number of packets reported for that interface...
> But then there is an
> discrepancy between WAN-ingress and LAN-egress (i.e. WAN-ingress is at
> ~400kbytes/s and LAN-egress on the host running bittorrent is
> ~220kbytes/s). Unfortunately I cannot see the bandwidth my cable-modem
> syncs at (I searched the web-interface but did not find it; it is a
> Comcast-provided Arris TG1682G).
With DOCSIS systems there is no clear and easy way to get hod on the relevant numbers (which would be the numbers your ISP plugged into the CMTS shaper and information about potentially allowed bursts).
>
> Noah, could you try and see what happens if instead of bittorrent a
> LAN-host downloads multiple files using TCP?
I would also be interested in the outcome of that test, please keep posting to the list.
Best Regards
Sebastian
>
> Thank you all for your feedback,
> George
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
More information about the Cake
mailing list