Re: [Bloat] TCP BBR paper is now generally available

[Neal Cardwell <ncardwell@google.com>]

Thanks for the report, Steinar. This is the first report we've had
like this, but it would be interesting to find out what's going on.

Even if you don't have time to apply the patches Eric mentions, it
would be hugely useful if the next time you have a slow transfer like
that you could post a link to a tcpdump packet capture (headers only
is best, say -s 120). Ideally the trace would capture a whole
connection, so we can see the wscale on the SYN exchange.

thanks,
neal


On Fri, Dec 2, 2016 at 6:31 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Fri, 2016-12-02 at 23:40 +0100, Steinar H. Gunderson wrote:
>> On Fri, Dec 02, 2016 at 05:22:23PM -0500, Neal Cardwell wrote:
>> > Of course, if we find important use cases that don't work with BBR, we will
>> > see what we can do to make BBR work well with them.
>>
>> I have one thing that I _wonder_ if could be BBR's fault: I run backup over
>> SSH. (That would be tar + gzip + ssh.) The first full backup after I rolled
>> out BBR on the server (the one sending the data) suddenly was very slow
>> (~50 Mbit/sec); there was plenty of free I/O, and neither tar nor gzip
>> (well, pigz) used a full core. My only remaining explanation would be that
>> somehow, BBR didn't deal well with the irregular stream of data coming from
>> tar. (A wget between the same machines at the same time gave 6-700 Mbit/sec.)
>>
>> I will not really blame BBR here, since I didn't take a tcpdump or have time
>> to otherwise debug properly (short of eliminating the other things I already
>> mentioned); most likely, it's something else. But if you've ever heard of
>> others with similar issues, consider this a second report. :-)
>>
>> /* Steinar */
>
> It would be interesting to get the chrono stats for the TCP flow, with
> an updated ss/iproute2 command and the kernel patches :
>
> efd90174167530c67a54273fd5d8369c87f9bd32 tcp: export sender limits chronographs to TCP_INFO
> b0f71bd3e190df827d25d7f19bf09037567f14b7 tcp: instrument how long TCP is limited by insufficient send buffer
> 5615f88614a47d2b802e1d14d31b623696109276 tcp: instrument how long TCP is limited by receive window
> 0f87230d1a6c253681550c6064715d06a32be73d tcp: instrument how long TCP is busy sending
> 05b055e89121394058c75dc354e9a46e1e765579 tcp: instrument tcp sender limits chronographs
>
>
>