From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 08D163B2C4 for ; Mon, 29 Feb 2016 07:35:40 -0500 (EST) Received: by mail-wm0-x231.google.com with SMTP id l68so60230078wml.0 for ; Mon, 29 Feb 2016 04:35:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tieto.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-transfer-encoding; bh=pwdy5J528XQ2KXqBDllVznQVluTxS0htSAQHGXg6rSA=; b=nSnJna+0t4DV1BBehl63jk21CWeEsipqCuA0WPpakcDjo7NmRy/PzsPS/c6gl1cCvK S0FTmyrqK7A8Sqi35YqIZDDBKNlI04gc7ZhH9gfMXGG/uTwGmBvCpDsJ42252teM6Bwh axj1jERuy/2JiWQoSZLweC4TFtG9lnnT03X4Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-transfer-encoding; bh=pwdy5J528XQ2KXqBDllVznQVluTxS0htSAQHGXg6rSA=; b=IN/K1UpXjQGt7Q4TFlHb86ze+l/0TypQu6zE55onQRwKzZR+KuMPyDHVtsy7U+FIkL W7z6YHgDrsLvjHG/gwRrMoFgI9oVYpkwCf6nTBTdnAykuaprW5ntplff+/yLY4SzYnHI 1ae1MWLW8la+B/RlRi0WI+lIiukCz8/zfWLeApiwVy7qxzTJ5Nihy5GTwuXTcin2xagu +kAIl6NHKaUpCMkJ4vqledmm77ultsmayLm1hEi3yJpkuQ4LymEcWTebBZPHFxVOGayY DgK0HI8Hj3A8iHci6z+DPA2LYlbPqjFI2ZSNRAXGBdxsLV3nN3G5bucbxxhNghgZjiYa gr/Q== X-Gm-Message-State: AD7BkJJJAaHXv1NNEmHsBlF6y/BqIjGbAB3zKMk0++akwxS7LI1WzrCNbTJmqVqHbxZReld97B7spirrrL5rkSU0fAsnjTX12hB6UMhKo5U+7OxJg6WqYDfuElXqU6OGIIG+d4u2DpF9aHF2A0JpLtD3BQ== MIME-Version: 1.0 X-Received: by 10.194.81.103 with SMTP id z7mr14234446wjx.25.1456749339185; Mon, 29 Feb 2016 04:35:39 -0800 (PST) Received: by 10.194.34.97 with HTTP; Mon, 29 Feb 2016 04:35:39 -0800 (PST) In-Reply-To: References: <1456492163-11437-1-git-send-email-michal.kazior@tieto.com> Message-ID: From: Michal Kazior To: Dave Taht Cc: make-wifi-fast@lists.bufferbloat.net, "codel@lists.bufferbloat.net" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-DomainID: tieto.com X-Mailman-Approved-At: Wed, 06 Apr 2016 13:11:42 -0400 Subject: Re: [Codel] [RFC/RFT] mac80211: implement fq_codel for software queuing X-BeenThere: codel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: CoDel AQM discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Mon, 29 Feb 2016 12:35:40 -0000 X-Original-Date: Mon, 29 Feb 2016 13:35:39 +0100 X-List-Received-Date: Mon, 29 Feb 2016 12:35:40 -0000 On 26 February 2016 at 23:20, Dave Taht wrote: > Dear Michal: > > Can you take a picture of your setup? I guess a diagram must do for now: .---------[G0] | [L0] [AP] [L4] [L1] [L2] [L3] * diagram skips testbed control plane * G0 is traffic generator - connected via ethernet to AP (AP bridges traffic) - running 3.16 * AP runs QCA99X0 (4 antenna) non-encrypted network * L0..L4 are laptops - running 4.3.0 * each has up to 3 QCA9337 (1 antenna) chips * total 10 clients - all connected to the AP * some of the chips are mounted on an Express Card adapters * some of the chips are mounted inside with mPCI-E -> M.2 adapters - antennas are put rogue-style through gaps in laptop' exterior * each client antenna is placed in ~0.5m away from the AP * client antennae are not uniformly placed with regard to each other (limited by pigtail lengths) * each client chip is run inside a QEMU VM with PCI-passthrough Let me know if you want to know more details. > Our intent is to continue to improve the flent test suite to be able > to generate repeatable tests, track relevant wifi behaviors and pull > relevant data back, graphed over time (of test) and time (over test > runs). A problem with udp flood tests is that tcp traffic is always > bidirectional (data vs acks), so a naive thought would be, that yes, > you should get half the bandwidth you get with a udp flood test. I don't see why you'd be doomed to get only half the bandwidth because of that? Sure, Wi-Fi is half-duplex but transmit time for ACKs is a lot smaller than transmit time for the data. Moreover you also have stuff like satellite links which have inherently long latency/pipes and large Bandwidth-Delay Product. You could think of Wi-Fi in a similar fashion (albeit it's more dynamic so it's not directly comparable). I'm not saying it should be the default though. > But in the age of aggregation that is not correct. > > It is my hope for us to join you on testing/evaluating the various > bits, but with so many patches (wonderfully, but suddenly) flying > around in loose formation ( can we start a lowlatency-wifi kernel tree > somewhere? - oy, there are so many other moving parts!), that's going > to take a bit. While we have some ath10k gear, the biggest testbeds > (karstad, san francisco, yurtlab) are all ath9k based. > > Some things you could do for us whilst we try to catch up. > > Take packet captures! - there are plenty of tcp experts on the codel list= . > > For single station tests: run a repeatable test series: rrul, rrul_be, > tcp_upload, tcp_download. Provide those flent.gz files. > rrul exercises 3 of the 4 802.11e queues on most systems. > rrul_be one queue > > Example: > #!/bin/sh > T=3D"some meaningful title like fq_codel_target_30ms_10meters-crazynewpat= ch-1" > S=3Dsome.netperf.server.nearby > F=3D"fent -x -l 60 " > TESTS=3D"rrul rrul_be tcp_upload tcp_download" > > for i in $TESTS > do > $F -H $S -t "$T" > done > > flent-gui *.gz > > If you are running tests overnight (recommended, wifi data is noisy so > are office environments), iterate on the $T-test number... > > You can also track remote queue lengths and stats with other flent > options. > My assumption however is that you are almost entirely > bypassing the qdisc queue now(?) and injecting things into a queue > that cannot be seen by userland? Yes. The patch uses IFF_NO_QUEUE (it would be dev->tx_queue_len=3D0 in pre-4.2 I think) so there are no qdiscs. Hence there's also no tx queue wake/stop logic performed. Userspace shouldn't see much of a difference because sockets still keep track of sk_buffs (and hence block on write/sendmsg when socket buffer limit is reached). Since the fq_drop() looks for elephant flows and head-drops them even if txq_limit limits is reached, it should work fine even without subqueue_stop/wake. > For playing with MU-mimo, the various rtt_fair tests in flent were our > starting point, which test anywhere from 1 to 4 stations. example > testing 2 stations with two tcp streams. > > rtt_fair4be -H station1 -H station2 -H station1 -H station2 > > The packet captures should be *fascinating* on that. > > Aircaps interesting also. > > Other variables to tweak: > > 0) Use the same driver on server and client. Then a reference driver. > 1) Disable codel entirely or give it a really big target/interval > (30ms, 300ms) to just look at the fq portion of the algorithm. > 2) enabling ECN on the tcps on server and client will give you a clear > idea as to when codel was kicking in vs packets being dropped > elsewhere on the packet captures. My current patch doesn't handle ECN. > 3) One of my biggest ongoing concerns with adapting codel in wifi has > been the impact of multicast on it - mdns-scan (along with any of the > above tests), or some other heavy mcast program in the background > (uftp is not bad). mu-mimo introduces new issues with sounding that I > don't think anyone understands at any detail yet. > Can wireshark or > some other tool "see" a sounding? Hmm.. NDP (null-data-packets) don't have any MAC payload to my knowledge which makes it kind of pointless to even report to the host. Even if it does it'd need some low-level RF data that is derived from receiving such packets. Radiotap isn't sufficient for that, I'm sure. Vendor radiotap could be used but I still don't know what info could/should be exposed for TxBF sounding. Otherwise there are is also sounding management frames for starting/controlling sounding (if I'm remembering right) so you should be - at least - able to see that sounding is being *attempted*. > 4) Distance and rate control. MCS4 was my basic rate for transmits > from stations for the longest time as that appeared to be the median > rate I'd got in various coffee shops... while I realize you have to > achieve peak throughput under ideal conditions, it's achieving good > overall performance in more abusive conditions... > > ... and ... > > 5) come to battlemesh with what you got. Sounds tempting but I can't promise anything. Anyway, thanks for all the tips! I'll play with flent and get back to you later. I've been preempted by other things for the time being.. Micha=C5=82