From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 0BAF62006AE for ; Sat, 19 Nov 2011 13:53:48 -0800 (PST) Received: by iaby12 with SMTP id y12so8409018iab.16 for ; Sat, 19 Nov 2011 13:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=beta; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-system-of-record; bh=TL9ZXZ8FX35Y7PDfUztul94lsi30mylirhZ78ut+8Ko=; b=P9ncAO5ajso+tT2cK/xupwaieyUpy/OAVqO6lSq6PUZ5PvDRoAH3UhK2GrDtMQVIh9 Gy39ByNh1AZDBCTvCSSg== Received: by 10.42.29.137 with SMTP id r9mr6372521icc.20.1321739628171; Sat, 19 Nov 2011 13:53:48 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.29.137 with SMTP id r9mr6372513icc.20.1321739628073; Sat, 19 Nov 2011 13:53:48 -0800 (PST) Received: by 10.231.40.6 with HTTP; Sat, 19 Nov 2011 13:53:47 -0800 (PST) In-Reply-To: References: Date: Sat, 19 Nov 2011 13:53:47 -0800 Message-ID: From: Tom Herbert To: Dave Taht Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-System-Of-Record: true X-Mailman-Approved-At: Tue, 22 Nov 2011 08:15:15 -0800 Cc: bloat Subject: Re: [Bloat] some (very good) preliminary results from fiddling with byte queue limits on 100Mbit ethernet X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Nov 2011 21:53:49 -0000 Thanks for trying this out Dave! > With byte queue limits at mtu*3 + the SFQ qdisc, latency under load > can be hammered > =A0down below 6ms when running at a 100Mbit line rate. No CBQ needed. > I'm hoping that we didn't have to set the BQL max_limit. I would guess that this might indicate some periodic spikes in interrupt latency (BQL will increase limit aggressively in that case). You might want to try adjusting the hold_time to a lower value. Also, disabling TSO might lower the limit. Without lowering the max_limit, what values so you see for limit and inflight? If you set min_limit to a really big number (effectively turn of BQL), what does inflight grow to? > Anyway, script could use improvement, and I'm busily patching BQL into > the ag71xx driver as I write. > Cool, I look forward to those results! > Sorry it's taken me so long to get to this since your bufferbloat > talks at linux plumbers. APPLAUSE. > It's looking like BQL + SFQ is an effective means of improving > fairness and reducing latency on drivers > that can support it. Even if they have large tx rings that the > hardware demands. > Great. I actually got back to looking at this a little last week. AFAICT the overhead of BQL is < 1% CPU and throughput (still need more testing to verify that). There are some (very) minor performance improvements that might be possible, but I don't have any major modifications pending at this point.