From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-x233.google.com (mail-oi0-x233.google.com [IPv6:2607:f8b0:4003:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id F23373B2A0 for ; Sat, 3 Dec 2016 08:04:21 -0500 (EST) Received: by mail-oi0-x233.google.com with SMTP id w63so295515899oiw.0 for ; Sat, 03 Dec 2016 05:04:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=lzksxY68T/ATjMu+hW2qq1QIl7AkAsJrUBMuHBgFSpw=; b=FF87ebDbos9Vy8/pylTPLCkaMKAYtVg6jFtFD9Hs28sJnmErfAz2p7ePNE9w7UmMiC YbpBiw/fkmRaASFx3tsIe+t7GkYTEkFGfavIgCytylIJgy5jUDiHGFNxD4366698GSfD tNrZtdNj1BCi9HRzlPBWgFfmRiuWk/LN/7UIEQNrpYWFccrOD8q35q+dkWcvKLcvsq0r Wmwy+hoXNRnlbq1+j3zrroQYgoaJWR8PqG/5G0NmNadRO89SVrWj8aIo1t5ImGms5zjJ 29fzCRLlrCZGhix4l7JAA9m+GA7UdgrW/xIl/CqGo4PAh5pwGU5Peanq6extO8q66bjH 9ieA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=lzksxY68T/ATjMu+hW2qq1QIl7AkAsJrUBMuHBgFSpw=; b=bkO7hr1nTrkLhYNz8NOKai6MvK4The8WxUmTOAmjsK8zDm3xkJD/qAI9Le68vhKmxr sOznvwUElYlb8l4eHXr9R2sQYBZP8Kz7cizLzb/zbDfi1k31exSQnDP42xNinWPgh5Z/ fFzTwhK2CEXCkk/s2K1uxlfGCgL5HFdcUE5eTCa31s12GsIuLMheotylRDwz4Rx0e+kj 8+haTNR6rlatIqI5A0dURYRin6ny25OLueMU7pg01Wu7EyK9FoQ1E7MG4mOxGJAh4CEv rTL+PzmBXIfhUnSeGRp9B6Ph0PCEqS4sLytfQibPt9ox8Ik9KLpWSNLDVY8zMn7yViBe 4Kkg== X-Gm-Message-State: AKaTC02UsOJd3mRFGXa662KfAzAkt82SbzjF7zbzpeH9XPEFcbdb66nUZbdGtdJu3hBPoMKfiv1NMJvvn6Aekn+g X-Received: by 10.202.239.84 with SMTP id n81mr24432133oih.94.1480770261345; Sat, 03 Dec 2016 05:04:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.202.73.195 with HTTP; Sat, 3 Dec 2016 05:03:50 -0800 (PST) In-Reply-To: <1480721486.18162.392.camel@edumazet-glaptop3.roam.corp.google.com> References: <56F6A3AB-3A47-4178-BEFF-04E3DC23B039@gmail.com> <20161202224006.GA5065@sesse.net> <1480721486.18162.392.camel@edumazet-glaptop3.roam.corp.google.com> From: Neal Cardwell Date: Sat, 3 Dec 2016 08:03:50 -0500 Message-ID: To: Eric Dumazet Cc: "Steinar H. Gunderson" , Jonathan Morton , "aqm@ietf.org" , bloat Content-Type: text/plain; charset=UTF-8 Subject: Re: [Bloat] TCP BBR paper is now generally available X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Dec 2016 13:04:22 -0000 Thanks for the report, Steinar. This is the first report we've had like this, but it would be interesting to find out what's going on. Even if you don't have time to apply the patches Eric mentions, it would be hugely useful if the next time you have a slow transfer like that you could post a link to a tcpdump packet capture (headers only is best, say -s 120). Ideally the trace would capture a whole connection, so we can see the wscale on the SYN exchange. thanks, neal On Fri, Dec 2, 2016 at 6:31 PM, Eric Dumazet wrote: > On Fri, 2016-12-02 at 23:40 +0100, Steinar H. Gunderson wrote: >> On Fri, Dec 02, 2016 at 05:22:23PM -0500, Neal Cardwell wrote: >> > Of course, if we find important use cases that don't work with BBR, we will >> > see what we can do to make BBR work well with them. >> >> I have one thing that I _wonder_ if could be BBR's fault: I run backup over >> SSH. (That would be tar + gzip + ssh.) The first full backup after I rolled >> out BBR on the server (the one sending the data) suddenly was very slow >> (~50 Mbit/sec); there was plenty of free I/O, and neither tar nor gzip >> (well, pigz) used a full core. My only remaining explanation would be that >> somehow, BBR didn't deal well with the irregular stream of data coming from >> tar. (A wget between the same machines at the same time gave 6-700 Mbit/sec.) >> >> I will not really blame BBR here, since I didn't take a tcpdump or have time >> to otherwise debug properly (short of eliminating the other things I already >> mentioned); most likely, it's something else. But if you've ever heard of >> others with similar issues, consider this a second report. :-) >> >> /* Steinar */ > > It would be interesting to get the chrono stats for the TCP flow, with > an updated ss/iproute2 command and the kernel patches : > > efd90174167530c67a54273fd5d8369c87f9bd32 tcp: export sender limits chronographs to TCP_INFO > b0f71bd3e190df827d25d7f19bf09037567f14b7 tcp: instrument how long TCP is limited by insufficient send buffer > 5615f88614a47d2b802e1d14d31b623696109276 tcp: instrument how long TCP is limited by receive window > 0f87230d1a6c253681550c6064715d06a32be73d tcp: instrument how long TCP is busy sending > 05b055e89121394058c75dc354e9a46e1e765579 tcp: instrument tcp sender limits chronographs > > >