Enabling Byte Queue Limits in the ath5k wireless driver

Enabling Byte Queue Limits in the ath5k wireless driver

[Anirudh Sivaraman]

Hi,

I am looking into enabling BQL for the ath5k driver, and was wondering
if anyone here knows of any prior efforts in this direction. In
particular, is BQL even a sensible strategy for wireless drivers?
Thank you in advance for any advice you may have in this regard.

Anirudh

Re: Enabling Byte Queue Limits in the ath5k wireless driver

[Simon Barber]

BQL does make sense, but TQL would be even better - 'Time Queue Limits'. 
Which would work based on an estimate of how much time each packet will 
take to send, and limiting the hardware queue to contain a total mount 
of 'time'. The purpose of the hardware queue is to mask interrupt and 
other latencies involved in refilling an empty queue - a time based 
phenomenon. Hence time is the best metric to control it - with fixed 
speed interfaced like ethernet bytes=time, but not so on wireless.

Simon

On 10/3/2013 11:49 AM, Anirudh Sivaraman wrote:
> Hi,
>
> I am looking into enabling BQL for the ath5k driver, and was wondering
> if anyone here knows of any prior efforts in this direction. In
> particular, is BQL even a sensible strategy for wireless drivers?
> Thank you in advance for any advice you may have in this regard.
>
> Anirudh
> _______________________________________________
> Bloat-devel mailing list
> Bloat-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat-devel

Re: Enabling Byte Queue Limits in the ath5k wireless driver

[Anirudh Sivaraman]

On Sat, Oct 5, 2013 at 11:39 PM, Simon Barber <simon@superduper.net> wrote:
> BQL does make sense, but TQL would be even better - 'Time Queue Limits'.
> Which would work based on an estimate of how much time each packet will take
> to send, and limiting the hardware queue to contain a total mount of 'time'.
> The purpose of the hardware queue is to mask interrupt and other latencies
> involved in refilling an empty queue - a time based phenomenon. Hence time
> is the best metric to control it - with fixed speed interfaced like ethernet
> bytes=time, but not so on wireless.
>

Thank you for your reply. TQL does make more sense for wireless at
least. However, assuming that someone implement a qdisc like CoDel and
attaches it to the wireless interface, doesn't TQL (or for that matter
BQL) lead to two possibly competing control loops (one at the qdisc,
and one at the device driver)?

Anirudh
> Simon
>
>
> On 10/3/2013 11:49 AM, Anirudh Sivaraman wrote:
>>
>> Hi,
>>
>> I am looking into enabling BQL for the ath5k driver, and was wondering
>> if anyone here knows of any prior efforts in this direction. In
>> particular, is BQL even a sensible strategy for wireless drivers?
>> Thank you in advance for any advice you may have in this regard.
>>
>> Anirudh
>> _______________________________________________
>> Bloat-devel mailing list
>> Bloat-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat-devel
>
>
> _______________________________________________
> Bloat-devel mailing list
> Bloat-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat-devel

Re: Enabling Byte Queue Limits in the ath5k wireless driver

[Simon Barber]

BQL or TQL are open-loop - there is no feedback. The amount of bytes or 
total packet transmission time in the queue is counted when packets 
enter and leave the queue, and a hard limit of total bytes or total time 
in queue is set. When I say time - I mean that on a packet entering the 
queue the total time the packet will take to transmit is calculated, and 
that time is added to a count to the total time currently in queue. When 
a packet has been transmitted the time is subtracted from the total time 
in queue. There would be no interference with CODEL running in front of 
this.

Simon

On 10/6/2013 6:31 AM, Anirudh Sivaraman wrote:
> On Sat, Oct 5, 2013 at 11:39 PM, Simon Barber <simon@superduper.net> wrote:
>> BQL does make sense, but TQL would be even better - 'Time Queue Limits'.
>> Which would work based on an estimate of how much time each packet will take
>> to send, and limiting the hardware queue to contain a total mount of 'time'.
>> The purpose of the hardware queue is to mask interrupt and other latencies
>> involved in refilling an empty queue - a time based phenomenon. Hence time
>> is the best metric to control it - with fixed speed interfaced like ethernet
>> bytes=time, but not so on wireless.
>>
> Thank you for your reply. TQL does make more sense for wireless at
> least. However, assuming that someone implement a qdisc like CoDel and
> attaches it to the wireless interface, doesn't TQL (or for that matter
> BQL) lead to two possibly competing control loops (one at the qdisc,
> and one at the device driver)?
>
> Anirudh
>> Simon
>>
>>
>> On 10/3/2013 11:49 AM, Anirudh Sivaraman wrote:
>>> Hi,
>>>
>>> I am looking into enabling BQL for the ath5k driver, and was wondering
>>> if anyone here knows of any prior efforts in this direction. In
>>> particular, is BQL even a sensible strategy for wireless drivers?
>>> Thank you in advance for any advice you may have in this regard.
>>>
>>> Anirudh
>>> _______________________________________________
>>> Bloat-devel mailing list
>>> Bloat-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>
>> _______________________________________________
>> Bloat-devel mailing list
>> Bloat-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat-devel

Re: Enabling Byte Queue Limits in the ath5k wireless driver

[Anirudh Sivaraman]

On Sun, Oct 6, 2013 at 11:01 AM, Simon Barber <simon@superduper.net> wrote:
> BQL or TQL are open-loop - there is no feedback. The amount of bytes or
> total packet transmission time in the queue is counted when packets enter
> and leave the queue, and a hard limit of total bytes or total time in queue
> is set. When I say time - I mean that on a packet entering the queue the
> total time the packet will take to transmit is calculated, and that time is
> added to a count to the total time currently in queue. When a packet has
> been transmitted the time is subtracted from the total time in queue. There
> would be no interference with CODEL running in front of this.
>

I thought, based on http://lwn.net/Articles/454390/, that the queue
length is dynamically adjusted by DQL/BQL/TQL. This is the device
driver control loop I was referring to. This queue length in turn
dictates when the dequeue function is called at the qdisc layer, where
CoDel is running. Please correct me if I am wrong here.

Anirudh

> Simon
>
>
> On 10/6/2013 6:31 AM, Anirudh Sivaraman wrote:
>>
>> On Sat, Oct 5, 2013 at 11:39 PM, Simon Barber <simon@superduper.net>
>> wrote:
>>>
>>> BQL does make sense, but TQL would be even better - 'Time Queue Limits'.
>>> Which would work based on an estimate of how much time each packet will
>>> take
>>> to send, and limiting the hardware queue to contain a total mount of
>>> 'time'.
>>> The purpose of the hardware queue is to mask interrupt and other
>>> latencies
>>> involved in refilling an empty queue - a time based phenomenon. Hence
>>> time
>>> is the best metric to control it - with fixed speed interfaced like
>>> ethernet
>>> bytes=time, but not so on wireless.
>>>
>> Thank you for your reply. TQL does make more sense for wireless at
>> least. However, assuming that someone implement a qdisc like CoDel and
>> attaches it to the wireless interface, doesn't TQL (or for that matter
>> BQL) lead to two possibly competing control loops (one at the qdisc,
>> and one at the device driver)?
>>
>> Anirudh
>>>
>>> Simon
>>>
>>>
>>> On 10/3/2013 11:49 AM, Anirudh Sivaraman wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am looking into enabling BQL for the ath5k driver, and was wondering
>>>> if anyone here knows of any prior efforts in this direction. In
>>>> particular, is BQL even a sensible strategy for wireless drivers?
>>>> Thank you in advance for any advice you may have in this regard.
>>>>
>>>> Anirudh
>>>> _______________________________________________
>>>> Bloat-devel mailing list
>>>> Bloat-devel@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>>
>>>
>>> _______________________________________________
>>> Bloat-devel mailing list
>>> Bloat-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>
>

Re: Enabling Byte Queue Limits in the ath5k wireless driver

[Simon Barber]

Indeed - but this is based on underruns while there is more to send - 
once it has established a length that does a good job of keeping the 
hardware fed it should not change much. This control loop will slowly 
catch up with the different speeds wireless packets are sent at (if a 
single link is in use, not multiple links) but it would be better 
controlled and also handle multiple links on one interface (e.g. Access 
point mode, or 11z) properly if packet duration was considered.

Simon

On 10/6/2013 9:36 AM, Anirudh Sivaraman wrote:
> On Sun, Oct 6, 2013 at 11:01 AM, Simon Barber <simon@superduper.net> wrote:
>> BQL or TQL are open-loop - there is no feedback. The amount of bytes or
>> total packet transmission time in the queue is counted when packets enter
>> and leave the queue, and a hard limit of total bytes or total time in queue
>> is set. When I say time - I mean that on a packet entering the queue the
>> total time the packet will take to transmit is calculated, and that time is
>> added to a count to the total time currently in queue. When a packet has
>> been transmitted the time is subtracted from the total time in queue. There
>> would be no interference with CODEL running in front of this.
>>
> I thought, based on http://lwn.net/Articles/454390/, that the queue
> length is dynamically adjusted by DQL/BQL/TQL. This is the device
> driver control loop I was referring to. This queue length in turn
> dictates when the dequeue function is called at the qdisc layer, where
> CoDel is running. Please correct me if I am wrong here.
>
> Anirudh
>
>> Simon
>>
>>
>> On 10/6/2013 6:31 AM, Anirudh Sivaraman wrote:
>>> On Sat, Oct 5, 2013 at 11:39 PM, Simon Barber <simon@superduper.net>
>>> wrote:
>>>> BQL does make sense, but TQL would be even better - 'Time Queue Limits'.
>>>> Which would work based on an estimate of how much time each packet will
>>>> take
>>>> to send, and limiting the hardware queue to contain a total mount of
>>>> 'time'.
>>>> The purpose of the hardware queue is to mask interrupt and other
>>>> latencies
>>>> involved in refilling an empty queue - a time based phenomenon. Hence
>>>> time
>>>> is the best metric to control it - with fixed speed interfaced like
>>>> ethernet
>>>> bytes=time, but not so on wireless.
>>>>
>>> Thank you for your reply. TQL does make more sense for wireless at
>>> least. However, assuming that someone implement a qdisc like CoDel and
>>> attaches it to the wireless interface, doesn't TQL (or for that matter
>>> BQL) lead to two possibly competing control loops (one at the qdisc,
>>> and one at the device driver)?
>>>
>>> Anirudh
>>>> Simon
>>>>
>>>>
>>>> On 10/3/2013 11:49 AM, Anirudh Sivaraman wrote:
>>>>> Hi,
>>>>>
>>>>> I am looking into enabling BQL for the ath5k driver, and was wondering
>>>>> if anyone here knows of any prior efforts in this direction. In
>>>>> particular, is BQL even a sensible strategy for wireless drivers?
>>>>> Thank you in advance for any advice you may have in this regard.
>>>>>
>>>>> Anirudh
>>>>> _______________________________________________
>>>>> Bloat-devel mailing list
>>>>> Bloat-devel@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>>>
>>>> _______________________________________________
>>>> Bloat-devel mailing list
>>>> Bloat-devel@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>

Re: Enabling Byte Queue Limits in the ath5k wireless driver

[Anirudh Sivaraman]

On Sun, Oct 6, 2013 at 5:03 PM, Simon Barber <simon@superduper.net> wrote:
> Indeed - but this is based on underruns while there is more to send - once
> it has established a length that does a good job of keeping the hardware fed
> it should not change much. This control loop will slowly catch up with the
> different speeds wireless packets are sent at (if a single link is in use,
> not multiple links) but it would be better controlled and also handle
> multiple links on one interface (e.g. Access point mode, or 11z) properly if
> packet duration was considered.
>

Ok, that makes a lot of sense. I guess I 'll try implementing BQL
first, followed by TQL, and see what I find.

Anirudh

> Simon
>
>
> On 10/6/2013 9:36 AM, Anirudh Sivaraman wrote:
>>
>> On Sun, Oct 6, 2013 at 11:01 AM, Simon Barber <simon@superduper.net>
>> wrote:
>>>
>>> BQL or TQL are open-loop - there is no feedback. The amount of bytes or
>>> total packet transmission time in the queue is counted when packets enter
>>> and leave the queue, and a hard limit of total bytes or total time in
>>> queue
>>> is set. When I say time - I mean that on a packet entering the queue the
>>> total time the packet will take to transmit is calculated, and that time
>>> is
>>> added to a count to the total time currently in queue. When a packet has
>>> been transmitted the time is subtracted from the total time in queue.
>>> There
>>> would be no interference with CODEL running in front of this.
>>>
>> I thought, based on http://lwn.net/Articles/454390/, that the queue
>> length is dynamically adjusted by DQL/BQL/TQL. This is the device
>> driver control loop I was referring to. This queue length in turn
>> dictates when the dequeue function is called at the qdisc layer, where
>> CoDel is running. Please correct me if I am wrong here.
>>
>> Anirudh
>>
>>> Simon
>>>
>>>
>>> On 10/6/2013 6:31 AM, Anirudh Sivaraman wrote:
>>>>
>>>> On Sat, Oct 5, 2013 at 11:39 PM, Simon Barber <simon@superduper.net>
>>>> wrote:
>>>>>
>>>>> BQL does make sense, but TQL would be even better - 'Time Queue
>>>>> Limits'.
>>>>> Which would work based on an estimate of how much time each packet will
>>>>> take
>>>>> to send, and limiting the hardware queue to contain a total mount of
>>>>> 'time'.
>>>>> The purpose of the hardware queue is to mask interrupt and other
>>>>> latencies
>>>>> involved in refilling an empty queue - a time based phenomenon. Hence
>>>>> time
>>>>> is the best metric to control it - with fixed speed interfaced like
>>>>> ethernet
>>>>> bytes=time, but not so on wireless.
>>>>>
>>>> Thank you for your reply. TQL does make more sense for wireless at
>>>> least. However, assuming that someone implement a qdisc like CoDel and
>>>> attaches it to the wireless interface, doesn't TQL (or for that matter
>>>> BQL) lead to two possibly competing control loops (one at the qdisc,
>>>> and one at the device driver)?
>>>>
>>>> Anirudh
>>>>>
>>>>> Simon
>>>>>
>>>>>
>>>>> On 10/3/2013 11:49 AM, Anirudh Sivaraman wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am looking into enabling BQL for the ath5k driver, and was wondering
>>>>>> if anyone here knows of any prior efforts in this direction. In
>>>>>> particular, is BQL even a sensible strategy for wireless drivers?
>>>>>> Thank you in advance for any advice you may have in this regard.
>>>>>>
>>>>>> Anirudh
>>>>>> _______________________________________________
>>>>>> Bloat-devel mailing list
>>>>>> Bloat-devel@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Bloat-devel mailing list
>>>>> Bloat-devel@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/bloat-devel
>>>
>>>
>