From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 9C18121F426; Fri, 27 Mar 2015 20:31:36 -0700 (PDT) Received: by obvd1 with SMTP id d1so21251945obv.0; Fri, 27 Mar 2015 20:31:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=r9wRIAavnm34JKkMB6c51vJJb5Vuy+y5i+ip5glCUp0=; b=KnKyZE98gj6fZtAFTYzZeqfqG1gL/jWC7qpu5wPRVYDFYWxhCS1eN1MVpXqjT+zzY/ b+e5GCjnUbZMbXGoUk2PkwBElzjMlDbwtxDa37eZeGGuUlfxUVAyRkA1L/teW9cXgATr pvZw2I167ouSSYVL7IpjiIV/htzGk/gHXZzkHTpjABhpqVGaI+PG+wygMBs12sKUeMk1 yNbsFwwssarmjbSZQWYb7F/r4Zfw+YPIpXrcZwdcJcI9BukE03/gAZAkOEex9+pUqboC xPxprytMCPMgX5OllHfXlFGJV9aO9lgJ0CfGm2bVdSdMunStbvqO9STUhKDffvIhdBVw Btkg== MIME-Version: 1.0 X-Received: by 10.60.123.83 with SMTP id ly19mr18384101oeb.8.1427513495445; Fri, 27 Mar 2015 20:31:35 -0700 (PDT) Received: by 10.202.51.66 with HTTP; Fri, 27 Mar 2015 20:31:35 -0700 (PDT) In-Reply-To: <5515A8DF.8050902@candelatech.com> References: <55147C8A.4030804@candelatech.com> <55157250.6030208@gmail.com> <5515A8DF.8050902@candelatech.com> Date: Fri, 27 Mar 2015 20:31:35 -0700 Message-ID: From: Dave Taht To: Isaac Konikoff Content-Type: multipart/alternative; boundary=047d7b5d5498cee976051250e216 Cc: codel , bloat , cerowrt-devel Subject: Re: [Cerowrt-devel] [Bloat] capturing packets and applying qdiscs X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2015 03:32:04 -0000 --047d7b5d5498cee976051250e216 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable wonderful dataset isaac! A lot to learn there and quite a bit I can explain, which might take me days to do with graphs and the like. But it's late, and unless you are planning on doing another test run I will defer. It is mildly easier to look at this stuff in bulk, so I did a wget -l 1- m http://candelatech.com/downloads/wifi-reports/trial1/ on the data. Quick top level notes rather than write a massive blog with graph entry.... -1) These are totally artificial tests, stressing out queue management. There are no winners, or losers per se', only data. Someday we can get to winners and losers, but we have a zillion interrelated variables to isolate and fix first. So consider this data a *baseline* for what wifi - at the highest rate possible - looks like today - and I'd dearly like some results that are below mcs4 on average also as a baseline.... Typical wifi traffic looks nothing like rrul, for example. rrul vs rrul_be is useful for showing how badly 802.11e queues actually work today, however= . 0) Pretty hard to get close to the underlying capability of the mac, isn't it? Plenty of problems besides queue management could exist, including running out of cpu.... 1) SFQ has a default packet limit of 128 packets which does not appear to be enough at these speeds. Bump it to 1000 for a more direct comparison to the other qdiscs. You will note a rather big difference in cwnd on your packet captures, and bandwidth usage more similar to pfifo_fast. I would expect, anyway. 2) I have generally felt that txops needed more of a "packing" approach to wedging packets into a txop rather than a pure sfq or drr approach, as losses tend to be bursty, and maximizing the number of flows in a txop a goodness. SFQ packs better than DRR. That said there are so many compensation stuff (like retries) getting in the way right now... 3) The SFQ results being better than the fq_codel results in several cases are also due in part to an interaction of the drr quantum and a not high enough target to compensate for wifi jitter. But in looking at SFQ you can't point to a lower latency and say that's "better" when you also have a much lower achieved bandwidth. So I would appreciate a run where the stations had a fq_codel quantum 300 and target 30ms. APs, on the other hand, would be better a larger (incalculable, but say 4500) quantum, a similar target, and a per dst filter rather than the full 5 tuple. On Fri, Mar 27, 2015 at 12:00 PM, Isaac Konikoff wrote: > Thanks for pointing out horst. > > I've been trying wireshark io graphs such as: > retry comparison: wlan.fc.retry=3D=3D0 (line) to wlan.fc.retry=3D=3D1 (i= mpulse) > beacon delays: wlan.fc.type_subtype=3D=3D0x08 AVG frame.time_delta_displ= ayed > > I've uploaded my pcap files, netperf-wrapper results and lanforge script > reports which have some aggregate graphs below all of the pie charts. The > pcap files with 64sta in the name correspond to the script reports. > > candelatech.com/downloads/wifi-reports/trial1 > > I'll upload more once I try the qdisc suggestions and I'll generate > comparison plots. > > Isaac > > > On 03/27/2015 10:21 AM, Aaron Wood wrote: > > > > On Fri, Mar 27, 2015 at 8:08 AM, Richard Smith > wrote: > >> Using horst I've discovered that the major reason our WiFi network sucks >> is because 90% of the packets are sent at the 6mbit rate. Most of the r= est >> show up in the 12 and 24mbit zone with a tiny fraction of them using the >> higher MCS rates. >> >> Trying to couple the radiotap info with the packet decryption to discove= r >> the sources of those low-bit rate packets is where I've been running int= o >> difficulty. I can see the what but I haven't had much luck on the why. >> >> I totally agree with you that tools other than wireshark for analyzing >> this seem to be non-existent. > > > Using the following filter in Wireshark should get you all that 6Mbps > traffic: > > radiotap.datarate =3D=3D 6 > > Then it's pretty easy to dig into what those are (by wifi frame-type, at > least). At my network, that's mostly broadcast traffic (AP beacons and > whatnot), as the corporate wifi has been set to use that rate as the > broadcast rate. > > without capturing the WPA exchange, the contents of the data frames > can't be seen, of course. > > -Aaron > > > > --=20 Dave T=C3=A4ht Let's make wifi fast, less jittery and reliable again! https://plus.google.com/u/0/107942175615993706558/posts/TVX3o84jjmb --047d7b5d5498cee976051250e216 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
wonderful dataset isaac! A l= ot to learn there and quite a bit I can explain, which might take me days t= o do with graphs and the like.

But it's late, and unless y= ou are planning on doing another test run I will defer.

It is = mildly easier to look at this stuff in bulk, so I did a wget -l 1- m http://candelatech.com/downloads/wifi-reports/trial1/ on the data.
Quick top level notes rather than write a massive blog with gra= ph entry....

-1) These are totally artificial = tests, stressing out queue management. There are no
winners, = or losers per se', only data. Someday we can get to winners and losers,=
but we have a zillion interrelated variables to isolate and = fix first. So consider this data a *baseline* for what wifi - at the highes= t rate possible - looks like today - and I'd dearly like some results t= hat are below mcs4 on average also as a baseline....

Typi= cal wifi traffic looks nothing like rrul, for example. rrul vs rrul_be is u= seful for showing how badly 802.11e queues actually work today, however.
0) Pretty hard to get close to the underlying capability of= the mac, isn't it? Plenty of problems besides queue management could e= xist, including running out of cpu....

1) SFQ has a default pa= cket limit of 128 packets which does not appear to be enough at these speed= s. Bump it to 1000 for a more direct comparison to the other qdiscs.
You will note a rather big difference in cwnd on your packet c= aptures, and bandwidth usage more similar to pfifo_fast. I would expect, an= yway.

2) I have generally felt that txops needed more of a &qu= ot;packing" approach to wedging packets into a txop rather than a pure= sfq or drr approach, as losses tend to be bursty, and maximizing the numbe= r of flows in a txop a goodness.=C2=A0 SFQ packs better than DRR.

Th= at said there are so many compensation stuff (like retries) getting in the = way right now...

3) The SFQ results being better than the fq_c= odel results in several cases are also due in part to an interaction of the= drr quantum and a not high enough target to compensate for wifi jitter.=C2= =A0

But in looking at SFQ you can't point to a lower latency an= d say that's "better" when you also have a much lower achieve= d bandwidth.

So I would appreciate a run where the stations had a fq= _codel quantum 300 and target 30ms. APs, on the other hand, would be better= a larger (incalculable, but say 4500) quantum, a similar target, and a per= dst filter rather than the full 5 tuple.



On Fri, Mar 27, 2015 at 12:00 PM, Isaac Konikoff <koniko= fi@candelatech.com> wrote:
=20 =20 =20
Thanks for pointing out horst.

I've been trying wireshark io graphs such as:
retry comparison:=C2=A0 wlan.fc.retry=3D=3D0 (line) to wlan.fc.retry=3D= =3D1 (impulse)
beacon delays:=C2=A0 wlan.fc.type_subtype=3D=3D0x08 AVG frame.time_delta_displayed

I've uploaded my pcap files, netperf-wrapper results and lanforge script reports which have some aggregate graphs below all of the pie charts. The pcap files with 64sta in the name correspond to the script reports.

candelatech.com/downloads/wifi-reports/trial1

I'll upload more once I try the qdisc suggestions and I'll gene= rate comparison plots.

Isaac


On 03/27/2015 10:21 AM, Aaron Wood wrote:


On Fri, Mar 27, 2015 at 8:08 AM, Richard Smith <smithbone@gmail.com> wrote:
Using horst I've discovered that the major reason our WiFi network sucks is because 90% of the packets are sent at the 6mbit rate.=C2=A0 Most of the rest show up in the 12 and 24mbit zone with a tiny fraction of them using the higher MCS rates.

Trying to couple the radiotap info with the packet decryption to discover the sources of those low-bit rate packets is where I've been running into difficulty.=C2=A0= I can see the what but I haven't had much luck on the why.

I totally agree with you that tools other than wireshark for analyzing this seem to be non-existent.

Using the following filter in Wireshark should get you all that 6Mbps traffic: =C2=A0

radiotap.datarate =3D=3D 6

Then it's pretty easy to dig into what those are (by wifi frame-type, at least).=C2=A0 At my network, that's m= ostly broadcast traffic (AP beacons and whatnot), as the corporate wifi has been set to use that rate as the broadcast rate.

without capturing the WPA exchange, the contents of the data frames can't be seen, of course.

-Aaron





--
Dave T=C3=A4ht
Let's make wifi fast, less jittery and reliab= le again!

https://plus.google.com/u/0/1079421= 75615993706558/posts/TVX3o84jjmb
--047d7b5d5498cee976051250e216--