From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.toke.dk (mail.toke.dk [52.28.52.200]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 258E13CB38 for ; Thu, 15 Nov 2018 12:24:27 -0500 (EST) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=toke.dk; s=20161023; t=1542302665; bh=+61GhQNjnU8wdyMTs/FDeoj8smhkEy+LIIT7JyOcQ8I=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=WLMS9fgBj7/v8CdJxutgr4scinP9Us1dkY6uB3ClTzI6CLvv5qyQZnJBC+2LF5HgK ux+LLl6G6Y6qqNjuUOGnZBFGpZaCH7300v/U6mIP1/m1bwFszCL+MWiDFWcDQ7qz/t 6t0dYkbjYJ+zu1thTtxucSt4bn+Qmoq6TBrewo58TAVGZ+pI0tOs0YO55SonK9Zogs KbxiUq8Adg3uuVf+VlNNfp4/YBF5oX/wPLg8ytZkIJIIxhdLu/FlK2955OxZKY8xMm 1LWZoBCGOivT2i+KhcMT7Gx6ORIWDcuk2afSdvrNT7l5qYZyXMZtwyRzQMLDl+z0EC yfNDQQlr1KXtg== To: Felix Fietkau , Rajkumar Manoharan , linux-wireless@vger.kernel.org, ath10k@lists.infradead.org Cc: make-wifi-fast@lists.bufferbloat.net In-Reply-To: <8e7847ff-4c88-10ae-2223-2fc7321641d9@nbd.name> References: <1542063113-22438-1-git-send-email-rmanohar@codeaurora.org> <1542063113-22438-4-git-send-email-rmanohar@codeaurora.org> <871s7nv9pl.fsf@toke.dk> <8e7847ff-4c88-10ae-2223-2fc7321641d9@nbd.name> Date: Thu, 15 Nov 2018 09:24:22 -0800 X-Clacks-Overhead: GNU Terry Pratchett Message-ID: <87sh02tfsp.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Make-wifi-fast] [PATCH v3 3/6] mac80211: Add airtime accounting and scheduling to TXQs X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Nov 2018 17:24:27 -0000 Felix Fietkau writes: > On 2018-11-14 18:40, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >>> This part doesn't really make much sense to me, but maybe I'm >>> misunderstanding how the code works. >>> Let's assume we have a driver like ath9k or mt76, which tries to keep a >>> number of aggregates in the hardware queue, and the hardware queue is >>> currently empty. >>> If the current txq entry is kept at the head of the schedule list, >>> wouldn't the code just pull from that one over and over again, until >>> enough packets are transmitted by the hardware and their tx status >>> processed? >>> It seems to me that while fairness is still preserved in the long run, >>> this could lead to rather bursty scheduling, which may not be >>> particularly latency friendly. >>=20 >> Yes, it'll be a bit more bursty when the hardware queue is completely >> empty. However, when a TX completion comes back, that will adjust the >> deficit of that sta and cause it to be rotated on the next dequeue. This >> obviously relies on the fact that the lower-level hardware queue is >> sufficiently shallow to not add a lot of latency. But we want that to be >> the case anyway. In practice, it works quite well for ath9k, but not so >> well for ath10k because it has a large buffer in firmware. >>=20 >> If we requeue the TXQ at the end of the list, a station that is taking >> up too much airtime will fail to be throttled properly, so the >> queue-at-head is kinda needed to ensure fairness... > Thanks for the explanation, that makes sense to me. I have an idea on > how to mitigate the burstiness within the driver. I'll write it down in > pseudocode, please let me know if you think that'll work. I don't think it will, unfortunately. For example, consider the case where there are two stations queued; one with a large negative deficit (say, -10ms), and one with a positive deficit. In this case, we really need to throttle the station with a negative deficit. But if the driver loops and caches txqs, we'll get something like the following: - First driver loop iteration: returns TXQ with positive deficit. - Second driver loop iteration: Only the negative-deficit TXQ is in the mac80211 list, so it will loop until that TXQ's deficit turns positive and return it. Because of this, the negative-deficit station won't be throttled, and we won't get fairness. How many frames will mt76 queue up below the driver point? I.e., how much burstiness are you expecting this will introduce on that driver? Taking a step back, it's clear that it would be good to be able to dequeue packets to multiple STAs at once (we need that for MU-MIMO on ath10k as well). However, I don't think we can do that with the round-robin fairness scheduler; so we are going to need a different algorithm. I *think* it may be possible to do this with a virtual-time scheduler, but I haven't sat down and worked out the details yet... -Toke