From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yk0-x235.google.com (mail-yk0-x235.google.com [IPv6:2607:f8b0:4002:c07::235]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 8011721FC16 for ; Wed, 29 Jul 2015 10:56:29 -0700 (PDT) Received: by ykax123 with SMTP id x123so14407270yka.1 for ; Wed, 29 Jul 2015 10:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-type; bh=c2QD1LtFsppgJSe8auQyeSnsOGMV/4L7GN1SU85gw90=; b=FixdC52iz5QtD7+6j+yAeKY2o/9i2QOtUqnxveBgQQRCeEfg1EP37SrZ8Cbre1UYq/ YPUF9+y8Z2RWoURtt7IywjapoQfEBeCdjm05ZDBBBzwRD/WP5ECC5xPvZHwLIRElXaNF F2PL3kdF8g3bhozFDgAPgH2Iwr0SQw6hKPMRw7teDM2+T+WgYMUeSNrD+lJleKWP3SCJ /7oauKRm9N81A2S8sild2eiyWV8bOIJFuvArf1+6hGp3HA+2b5YuheQT0GBkub0BjeDG O95rzJMwut5XLxwC3QHmNoulRbfa1xzo/sUJFnDpBEyxDjGwKJeu3UsL1Dg2XoG/wm3y Fqrw== X-Received: by 10.129.91.87 with SMTP id p84mr47388445ywb.95.1438192588116; Wed, 29 Jul 2015 10:56:28 -0700 (PDT) MIME-Version: 1.0 References: <55B8B7EE.3070401@gmail.com> <55B8BABF.8040708@gmail.com> In-Reply-To: From: Rosen Penev Date: Wed, 29 Jul 2015 17:56:18 +0000 Message-ID: To: Dave Taht , David Lang Content-Type: multipart/alternative; boundary=001a114c7192555a02051c074e57 Cc: bloat Subject: Re: [Bloat] Kirkwood BQL? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jul 2015 17:56:57 -0000 --001a114c7192555a02051c074e57 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable My main question was why BQL support is not in the driver. I've seen it listed on the bloated drivers page for a long time but no real explanation why it remains so. I guess no interested party has the equipment to test and verify if it works correctly. I will try to compile my own kernel with the patch applied( current one has no codel compiled in) and see if any issues occur. I know that TSO support is totally broken with this driver (see: http://archlinuxarm.org/forum/viewtopic.php?f=3D9&t=3D7692&start=3D20= ). If only it didn't take a day to compile :\. On Wed, Jul 29, 2015, 10:42 Dave Taht wrote: > On Wed, Jul 29, 2015 at 7:07 PM, David Lang wrote: > > On Wed, 29 Jul 2015, Alan Jenkins wrote: > > > >> On 29/07/15 12:24, Alan Jenkins wrote: > >>> > >>> On 29/07/15 05:32, Rosen Penev wrote: > >>>> > >>>> Anyone know what the situation is with kirkwood and BQL? I found a > >>>> patch for it but have no idea if there are any issues. > >>>> > >>>> I have such a system but have no idea how to ascertain the efficacy = of > >>>> BQL. > >>> > >>> > >>> To the latter: > >>> > >>> BQL works for transmissions that reach the full line rate (e.g. for > >>> 1000MB ethernet). It limits the queue that builds in the > driver/device to > >>> the minimum they need. Then queue mostly builds in the generic > networking > >>> stack, where it can be managed effectively e.g. by fq_codel. > >>> > >>> So a simple efficacy test is to run a transmission at full speed, and > >>> monitor latency (ping) at the same time. Just make sure the device > qdisc is > >>> set to fq_codel. fq_codel effectively prioritizes ping, so the > difference > >>> will be very easy to see. > >>> > >>> I don't know if there's any corner cases that want testing as well. > > > > > > BQL adjusts the number of packets that can be queued based on their > size, so > > you can have far more 64 byte packets queued than you can have 1500 byt= e > > packets. > > > > do a ping flood of your network with different packet sizes and look at > the > > queue lengths that are allowed, the queue length should be much higher > with > > small packets. > > > >>> BQL can be disabled at runtime for comparison testing: > >>> http://lists.openwall.net/netdev/2011/12/01/112 > >>> > >>> There's a BQL tool to see it working graphically (using readouts from > the > >>> same sysfs directory): > >>> https://github.com/ffainelli/bqlmon > >>> > >>> My Kirkwood setup at home is weak, I basically never reach full link > >>> speed. So this might be somewhat academic unless you set the link > speed to > >>> 100 or 10 using the ethtool command. (It seems like a good idea to > test > >>> those speeds even if you can do better though). You probably also > want to > >>> start with offloads (tso, gso, gro) disabled using ethtool, because > they > >>> aggregate packets. > >>> > >> > >> a quick test with a 100M setting, connected to gigabit switch, and fle= nt > >> tcp_download, shows ping under load increases to about 8ms. Conclusion= : > the > >> Debian kirkwood kernel probably isn't doing BQL for me :). > > Wrong way I think. Try tcp_upload. > > > > > 8ms of latency under load is doing very well. what are you expecting? > > > > David Lang > > > > > >>> Flent can do this test and generate pretty graphs, including a time > >>> series (plot type "all_scaled") and frequency distribution for the pi= ng > >>> ("ping_cdf"). Flent is a frontend to the netperf network performance > >>> tester. You could use a directly connected laptop and run your own > netperf > >>> server (netserver command). You'll need to set up static IPs on both > ends > >>> for the duration... if headless then make sure you have an alternativ= e > >>> console access :). > >>> > >>> The normal Flent test is RRUL, which is two-way. tcp_2up would be > >>> better, to avoid testing both end's BQL at the same time. If you wan= t > to > >>> run tcp_2up the other way round, so you only need netserver on the > ARM, try > >>> using '--swap-up-down'. > >>> > >>> Alan > >> > >> > >> _______________________________________________ > >> Bloat mailing list > >> Bloat@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/bloat > >> > > _______________________________________________ > > Bloat mailing list > > Bloat@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/bloat > > > > -- > Dave T=C3=A4ht > worldwide bufferbloat report: > http://www.dslreports.com/speedtest/results/bufferbloat > And: > What will it take to vastly improve wifi for everyone? > https://plus.google.com/u/0/explore/makewififast > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --001a114c7192555a02051c074e57 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

My main question was why BQL support is not in the driver. I= 've seen it listed on the bloated drivers page for a long time but no r= eal explanation why it remains so. I guess no interested party has the equi= pment to test and verify if it works correctly. I will try to compile my ow= n kernel with the patch applied( current one has no codel compiled in) and = see if any issues occur. I know that TSO support is totally broken with thi= s driver (see: http://archlinuxarm.org/forum/viewtopic.php?f= =3D9&t=3D7692&start=3D20 ). If only it didn't take a day to= compile :\.


On Wed, Jul 29, 2015, 10:42= =C2=A0Dave Taht <dave.taht@gmail.= com> wrote:
On Wed, Jul 29, = 2015 at 7:07 PM, David Lang <david@lang.hm> wrote:
> On Wed, 29 Jul 2015, Alan Jenkins wrote:
>
>> On 29/07/15 12:24, Alan Jenkins wrote:
>>>
>>> On 29/07/15 05:32, Rosen Penev wrote:
>>>>
>>>> Anyone know what the situation is with kirkwood and BQL? I= found a
>>>> patch for it but have no idea if there are any issues.
>>>>
>>>> I have such a system but have no idea how to ascertain the= efficacy of
>>>> BQL.
>>>
>>>
>>> To the latter:
>>>
>>> BQL works for transmissions that reach the full line rate (e.g= . for
>>> 1000MB ethernet).=C2=A0 It limits the queue that builds in the= driver/device to
>>> the minimum they need.=C2=A0 Then queue mostly builds in the g= eneric networking
>>> stack, where it can be managed effectively e.g. by fq_codel. >>>
>>> So a simple efficacy test is to run a transmission at full spe= ed, and
>>> monitor latency (ping) at the same time.=C2=A0 Just make sure = the device qdisc is
>>> set to fq_codel.=C2=A0 fq_codel effectively prioritizes ping, = so the difference
>>> will be very easy to see.
>>>
>>> I don't know if there's any corner cases that want tes= ting as well.
>
>
> BQL adjusts the number of packets that can be queued based on their si= ze, so
> you can have far more 64 byte packets queued than you can have 1500 by= te
> packets.
>
> do a ping flood of your network with different packet sizes and look a= t the
> queue lengths that are allowed, the queue length should be much higher= with
> small packets.
>
>>> BQL can be disabled at runtime for comparison testing:
>>> http://lists.openwall.net/netdev/2011/12= /01/112
>>>
>>> There's a BQL tool to see it working graphically (using re= adouts from the
>>> same sysfs directory):
>>> https://github.com/ffainelli/bqlmon
>>>
>>> My Kirkwood setup at home is weak, I basically never reach ful= l link
>>> speed. So this might be somewhat academic unless you set the l= ink speed to
>>> 100 or 10 using the ethtool command.=C2=A0 (It seems like a go= od idea to test
>>> those speeds even if you can do better though).=C2=A0 You prob= ably also want to
>>> start with offloads (tso, gso, gro) disabled using ethtool, be= cause they
>>> aggregate packets.
>>>
>>
>> a quick test with a 100M setting, connected to gigabit switch, and= flent
>> tcp_download, shows ping under load increases to about 8ms. Conclu= sion: the
>> Debian kirkwood kernel probably isn't doing BQL for me :).

Wrong way I think. Try tcp_upload.

>
> 8ms of latency under load is doing very well. what are you expecting?<= br> >
> David Lang
>
>
>>> Flent can do this test and generate pretty graphs, including a= time
>>> series (plot type "all_scaled") and frequency distri= bution for the ping
>>> ("ping_cdf").=C2=A0 Flent is a frontend to the netpe= rf network performance
>>> tester.=C2=A0 You could use a directly connected laptop and ru= n your own netperf
>>> server (netserver command).=C2=A0 You'll need to set up st= atic IPs on both ends
>>> for the duration... if headless then make sure you have an alt= ernative
>>> console access :).
>>>
>>> The normal Flent test is RRUL, which is two-way.=C2=A0 tcp_2up= would be
>>> better, to avoid testing both end's BQL at the same time.= =C2=A0 If you want to
>>> run tcp_2up the other way round, so you only need netserver on= the ARM, try
>>> using '--swap-up-down'.
>>>
>>> Alan
>>
>>
>> _______________________________________________
>> Bloat mailing list
>> B= loat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
> _______________________________________________
> Bloat mailing list
> Bloat= @lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat


--
Dave T=C3=A4ht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/b= ufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast=
_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
--001a114c7192555a02051c074e57--