CoDel AQM discussions
 help / color / mirror / Atom feed
From: Roman Yeryomin <leroi.lists@gmail.com>
To: Dave Taht <dave.taht@gmail.com>
Cc: "Rajkumar Manoharan" <rmanohar@codeaurora.org>,
	"Michal Kazior" <michal.kazior@tieto.com>,
	"Jesper Dangaard Brouer" <brouer@redhat.com>,
	"Felix Fietkau" <nbd@nbd.name>,
	"Jonathan Morton" <chromatix99@gmail.com>,
	"codel@lists.bufferbloat.net" <codel@lists.bufferbloat.net>,
	ath10k <ath10k@lists.infradead.org>,
	make-wifi-fast@lists.bufferbloat.net,
	"Rafał Miłecki" <zajec5@gmail.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"OpenWrt Development List" <openwrt-devel@lists.openwrt.org>
Subject: Re: [Codel] OpenWRT wrong adjustment of fq_codel defaults (Was: fq_codel_drop vs a udp flood)
Date: Mon, 16 May 2016 22:46:59 +0300	[thread overview]
Message-ID: <CACiydbLK0kJZOUb+L3HH7FFv=xKVbH1MRd-7aTC39UPgwfR4VA@mail.gmail.com> (raw)
In-Reply-To: <CAA93jw7=FRhj+oZ8LT0Oxn2Lb3YFL-u5yZr7mi8bc2HZWzLxvg@mail.gmail.com>

On 16 May 2016 at 19:04, Dave Taht <dave.taht@gmail.com> wrote:
> On Mon, May 16, 2016 at 1:14 AM, Roman Yeryomin <leroi.lists@gmail.com> wrote:
>> On 16 May 2016 at 01:34, Roman Yeryomin <leroi.lists@gmail.com> wrote:
>>> On 6 May 2016 at 22:43, Dave Taht <dave.taht@gmail.com> wrote:
>>>> On Fri, May 6, 2016 at 11:56 AM, Roman Yeryomin <leroi.lists@gmail.com> wrote:
>>>>> On 6 May 2016 at 21:43, Roman Yeryomin <leroi.lists@gmail.com> wrote:
>>>>>> On 6 May 2016 at 15:47, Jesper Dangaard Brouer <brouer@redhat.com> wrote:
>>>>>>>
>>>>>>> I've created a OpenWRT ticket[1] on this issue, as it seems that someone[2]
>>>>>>> closed Felix'es OpenWRT email account (bad choice! emails bouncing).
>>>>>>> Sounds like OpenWRT and the LEDE https://www.lede-project.org/ project
>>>>>>> is in some kind of conflict.
>>>>>>>
>>>>>>> OpenWRT ticket [1] https://dev.openwrt.org/ticket/22349
>>>>>>>
>>>>>>> [2] http://thread.gmane.org/gmane.comp.embedded.openwrt.devel/40298/focus=40335
>>>>>>
>>>>>> OK, so, after porting the patch to 4.1 openwrt kernel and playing a
>>>>>> bit with fq_codel limits I was able to get 420Mbps UDP like this:
>>>>>> tc qdisc replace dev wlan0 parent :1 fq_codel flows 16 limit 256
>>>>>
>>>>> Forgot to mention, I've reduced drop_batch_size down to 32
>>>>
>>>> 0) Not clear to me if that's the right line, there are 4 wifi queues,
>>>> and the third one
>>>> is the BE queue.
>>>
>>> That was an example, sorry, should have stated that. I've applied same
>>> settings to all 4 queues.
>>>
>>>> That is too low a limit, also, for normal use. And:
>>>> for the purpose of this particular UDP test, flows 16 is ok, but not
>>>> ideal.
>>>
>>> I played with different combinations, it doesn't make any
>>> (significant) difference: 20-30Mbps, not more.
>>> What numbers would you propose?
>>>
>>>> 1) What's the tcp number (with a simultaneous ping) with this latest patchset?
>>>> (I care about tcp performance a lot more than udp floods - surviving a
>>>> udp flood yes, performance, no)
>>>
>>> During the test (both TCP and UDP) it's roughly 5ms in average, not
>>> running tests ~2ms. Actually I'm now wondering if target is working at
>>> all, because I had same result with target 80ms..
>>> So, yes, latency is good, but performance is poor.
>>>
>>>> before/after?
>>>>
>>>> tc -s qdisc show dev wlan0 during/after results?
>>>
>>> during the test:
>>>
>>> qdisc mq 0: root
>>>  Sent 1600496000 bytes 1057194 pkt (dropped 1421568, overlimits 0 requeues 17)
>>>  backlog 1545794b 1021p requeues 17
>>> qdisc fq_codel 8001: parent :1 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 8002: parent :2 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 8003: parent :3 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 1601271168 bytes 1057706 pkt (dropped 1422304, overlimits 0 requeues 17)
>>>  backlog 1541252b 1018p requeues 17
>>>   maxpacket 1514 drop_overlimit 1422304 new_flow_count 35 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 1
>>> qdisc fq_codel 8004: parent :4 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 0
>>>
>>>
>>> after the test (60sec):
>>>
>>> qdisc mq 0: root
>>>  Sent 3084996052 bytes 2037744 pkt (dropped 2770176, overlimits 0 requeues 28)
>>>  backlog 0b 0p requeues 28
>>> qdisc fq_codel 8001: parent :1 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 8002: parent :2 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 8003: parent :3 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 3084996052 bytes 2037744 pkt (dropped 2770176, overlimits 0 requeues 28)
>>>  backlog 0b 0p requeues 28
>>>   maxpacket 1514 drop_overlimit 2770176 new_flow_count 64 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 1
>>> qdisc fq_codel 8004: parent :4 limit 1024p flows 16 quantum 1514
>>> target 80.0ms ce_threshold 32us interval 100.0ms ecn
>>>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>   new_flows_len 0 old_flows_len 0
>>>
>>>
>>>> IF you are doing builds for the archer c7v2, I can join in on this... (?)
>>>
>>> I'm not but I have c7 somewhere, so I can do a build for it and also
>>> test, so we are on the same page.
>>>
>>>> I did do a test of the ath10k "before", fq_codel *never engaged*, and
>>>> tcp induced latencies under load, e at 100mbit, cracked 600ms, while
>>>> staying flat (20ms) at 100mbit. (not the same patches you are testing)
>>>> on x86. I have got tcp 300Mbit out of an osx box, similar latency,
>>>> have yet to get anything more on anything I currently have
>>>> before/after patchsets.
>>>>
>>>> I'll go add flooding to the tests, I just finished a series comparing
>>>> two different speed stations and life was good on that.
>>>>
>>>> "before" - fq_codel never engages, we see seconds of latency under load.
>>>>
>>>> root@apu2:~# tc -s qdisc show dev wlp4s0
>>>> qdisc mq 0: root
>>>>  Sent 8570563893 bytes 6326983 pkt (dropped 0, overlimits 0 requeues 0)
>>>>  backlog 0b 0p requeues 0
>>>> qdisc fq_codel 0: parent :1 limit 10240p flows 1024 quantum 1514
>>>> target 5.0ms interval 100.0ms ecn
>>>>  Sent 2262 bytes 17 pkt (dropped 0, overlimits 0 requeues 0)
>>>>  backlog 0b 0p requeues 0
>>>>   maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>>   new_flows_len 0 old_flows_len 0
>>>> qdisc fq_codel 0: parent :2 limit 10240p flows 1024 quantum 1514
>>>> target 5.0ms interval 100.0ms ecn
>>>>  Sent 220486569 bytes 152058 pkt (dropped 0, overlimits 0 requeues 0)
>>>>  backlog 0b 0p requeues 0
>>>>   maxpacket 18168 drop_overlimit 0 new_flow_count 1 ecn_mark 0
>>>>   new_flows_len 0 old_flows_len 1
>>>> qdisc fq_codel 0: parent :3 limit 10240p flows 1024 quantum 1514
>>>> target 5.0ms interval 100.0ms ecn
>>>>  Sent 8340546509 bytes 6163431 pkt (dropped 0, overlimits 0 requeues 0)
>>>>  backlog 0b 0p requeues 0
>>>>   maxpacket 68130 drop_overlimit 0 new_flow_count 120050 ecn_mark 0
>>>>   new_flows_len 1 old_flows_len 3
>>>> qdisc fq_codel 0: parent :4 limit 10240p flows 1024 quantum 1514
>>>> target 5.0ms interval 100.0ms ecn
>>>>  Sent 9528553 bytes 11477 pkt (dropped 0, overlimits 0 requeues 0)
>>>>  backlog 0b 0p requeues 0
>>>>   maxpacket 66 drop_overlimit 0 new_flow_count 1 ecn_mark 0
>>>>   new_flows_len 1 old_flows_len 0
>>>>   ```
>>>>
>>>>
>>>>>> This is certainly better than 30Mbps but still more than two times
>>>>>> less than before (900).
>>>>
>>>> The number that I still am not sure we got is that you were sending
>>>> 900mbit udp and recieving 900mbit on the prior tests?
>>>
>>> 900 was sending, AP POV (wifi client is downloading)
>>>
>>>>>> TCP also improved a little (550 to ~590).
>>>>
>>>> The limit is probably a bit low, also.  You might want to try target
>>>> 20ms as well.
>>>
>>> I've tried limit up to 1024 and target up to 80ms
>>>
>>>>>>
>>>>>> Felix, others, do you want to see the ported patch, maybe I did something wrong?
>>>>>> Doesn't look like it will save ath10k from performance regression.
>>>>
>>>> what was tcp "before"? (I'm sorry, such a long thread)
>>>
>>> 750Mbps
>>
>> Michal, after retesting with your patch (sorry, it was late yesterday,
>> confused compat-wireless archives) I saw the difference.
>> So the progress looks like this (all with fq_codel flows 16 limit 1024
>> target 20ms):
>> no patches: 380Mbps UDP, 550 TCP
>> Eric's (fq_codel drop) patch: 420Mbps UDP, 590 TCP (+40Mbps), latency
>> 5-6ms during test
>> Michal's (improve tx scheduling) patch: 580Mbps UDP, 660 TCP, latency
>> up to 30-40ms during test
>> after Rajkumar's proposal to "try without registering wake_tx_queue
>> callback": 820Mbps UDP, 690 TCP.
>
> And the simultaneous ping on the last test was?

same as previous: 30-40ms

Regards,
Roman

  reply	other threads:[~2016-05-16 19:47 UTC|newest]

Thread overview: 73+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-05-01  3:41 [Codel] fq_codel_drop vs a udp flood Dave Taht
2016-05-01  4:46 ` [Codel] [Make-wifi-fast] " Jonathan Morton
2016-05-01  5:08 ` [Codel] " Ben Greear
2016-05-01  5:23   ` Dave Taht
2016-05-01 14:47     ` [Codel] [Make-wifi-fast] " dpreed
     [not found]       ` <CAJq5cE2woA3yb6i_7NLPpxjzvhsVk5uL8BnSTAY7Lp-M0KiPNg@mail.gmail.com>
     [not found]         ` <CAJq5cE2K0yrz6ALAoKWu23RSJZX9Y_P7Mqcy9ba8e-L3AVhOaA@mail.gmail.com>
2016-05-01 15:51           ` Jonathan Morton
     [not found]       ` <CACiydb+nNEWfO8XS63MnrW0LazORAGX+L+fF1xZ2_2sCLeRAsg@mail.gmail.com>
2016-05-02 18:40         ` Dave Taht
2016-05-02 20:17           ` Isaac Konikoff
2016-05-05 13:55           ` Roman Yeryomin
2016-05-05 14:55             ` Roman Yeryomin
2016-05-02 19:47         ` David Lang
2016-05-01 17:59 ` [Codel] " Eric Dumazet
2016-05-01 18:20   ` Jonathan Morton
2016-05-01 18:46     ` Eric Dumazet
2016-05-01 19:55       ` Eric Dumazet
2016-05-02  7:47         ` Jesper Dangaard Brouer
2016-05-01 20:35       ` Jonathan Morton
2016-05-01 20:55         ` Eric Dumazet
     [not found]           ` <CACiydbKUu11=zWitkDha0ddgk1-G_Z4-e1+=9ky776VktF5HHg@mail.gmail.com>
2016-05-02 15:07             ` Eric Dumazet
     [not found]               ` <CACiydbKeKUENncrc-NmYRcku-DGVeGqqzYMqsCqKdxPsR7yUOQ@mail.gmail.com>
2016-05-02 16:14                 ` Eric Dumazet
2016-05-02 17:08                   ` Dave Taht
2016-05-02 17:44                     ` Eric Dumazet
2016-05-05 14:32                     ` Roman Yeryomin
2016-05-05 14:53                   ` Roman Yeryomin
2016-05-05 15:32                     ` Dave Taht
2016-05-05 16:07                       ` Roman Yeryomin
2016-05-05 16:59                         ` Jonathan Morton
2016-05-05 17:39                           ` Roman Yeryomin
2016-05-05 18:16                             ` Dave Taht
2016-05-05 18:33                           ` Dave Taht
2016-05-05 16:12                     ` Eric Dumazet
2016-05-05 16:25                       ` Roman Yeryomin
2016-05-05 16:42                         ` Roman Yeryomin
2016-05-06 10:55                           ` Roman Yeryomin
2016-05-05 19:23                         ` Eric Dumazet
2016-05-05 19:41                           ` Dave Taht
2016-05-06  8:41                             ` moeller0
2016-05-06 11:33                               ` Jesper Dangaard Brouer
2016-05-06 11:46                                 ` moeller0
2016-05-06 13:25                                   ` Eric Dumazet
2016-05-06 15:25                                     ` moeller0
2016-05-06 15:58                                       ` Eric Dumazet
2016-05-06 16:30                                         ` moeller0
2016-05-06  9:42                           ` [Codel] OpenWRT wrong adjustment of fq_codel defaults (Was: fq_codel_drop vs a udp flood) Jesper Dangaard Brouer
2016-05-06 12:47                             ` Jesper Dangaard Brouer
2016-05-06 18:43                               ` Roman Yeryomin
2016-05-06 18:56                                 ` Roman Yeryomin
2016-05-06 19:43                                   ` Dave Taht
2016-05-15 22:34                                     ` Roman Yeryomin
2016-05-15 23:07                                       ` Eric Dumazet
2016-05-15 23:27                                         ` Roman Yeryomin
2016-05-16  8:12                                       ` [Codel] [Make-wifi-fast] " David Lang
2016-05-16  8:26                                         ` Roman Yeryomin
2016-05-16  8:46                                           ` David Lang
2016-05-16 10:34                                             ` [Codel] [OpenWrt-Devel] " Sebastian Moeller
2016-05-16  8:14                                       ` [Codel] " Roman Yeryomin
2016-05-16 14:23                                         ` [Codel] [Make-wifi-fast] " Eric Dumazet
2016-05-16 16:04                                         ` [Codel] " Dave Taht
2016-05-16 19:46                                           ` Roman Yeryomin [this message]
     [not found]                             ` <572DBC21.10209@darbyshire-bryant.me.uk>
2016-05-15 22:47                               ` Roman Yeryomin
2016-05-03  2:26     ` [Codel] fq_codel_drop vs a udp flood Dave Taht
2016-05-03  5:21       ` Dave Taht
2016-05-03 12:39         ` Agarwal, Anil
2016-05-03 12:50           ` Agarwal, Anil
2016-05-03 13:35             ` Eric Dumazet
2016-05-03 15:37               ` Agarwal, Anil
2016-05-03 17:37               ` Dave Taht
2016-05-03 17:54                 ` Eric Dumazet
2016-05-03 18:11                   ` Dave Taht
2016-05-01 18:26   ` Dave Taht
2016-05-01 22:30     ` Eric Dumazet
     [not found]   ` <CACiydbL0K2fQT4EXm4LjDaxDLC+AVj9szTpyHq2iUwuPVNNz+g@mail.gmail.com>
2016-05-02 15:04     ` Eric Dumazet
     [not found] ` <CACiydbJStMOr0HKpw68zxDPgQRdwjR88aDis9fv12umzKAvWqw@mail.gmail.com>
2016-05-02 15:01   ` [Codel] [Make-wifi-fast] " Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/codel.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CACiydbLK0kJZOUb+L3HH7FFv=xKVbH1MRd-7aTC39UPgwfR4VA@mail.gmail.com' \
    --to=leroi.lists@gmail.com \
    --cc=ath10k@lists.infradead.org \
    --cc=brouer@redhat.com \
    --cc=chromatix99@gmail.com \
    --cc=codel@lists.bufferbloat.net \
    --cc=dave.taht@gmail.com \
    --cc=make-wifi-fast@lists.bufferbloat.net \
    --cc=michal.kazior@tieto.com \
    --cc=nbd@nbd.name \
    --cc=netdev@vger.kernel.org \
    --cc=openwrt-devel@lists.openwrt.org \
    --cc=rmanohar@codeaurora.org \
    --cc=zajec5@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox