From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from eu1sys200aog107.obsmtp.com (eu1sys200aog107.obsmtp.com [207.126.144.123]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 2FD0C20170C for ; Sat, 7 Apr 2012 04:54:37 -0700 (PDT) Received: from mail.la.pnsol.com ([89.145.213.110]) (using TLSv1) by eu1sys200aob107.postini.com ([207.126.147.11]) with SMTP ID DSNKT4Aq9/SHppz6+/MKXw8RKymg0PaFw33I@postini.com; Sat, 07 Apr 2012 11:54:37 UTC Received: from ba6-office.pnsol.com ([172.20.5.199]) by mail.la.pnsol.com with esmtp (Exim 4.63) (envelope-from ) id 1SGUDu-0003vq-Mv; Sat, 07 Apr 2012 12:54:30 +0100 Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Neil Davies In-Reply-To: <20120406213725.GA12641@uio.no> Date: Sat, 7 Apr 2012 12:54:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120406213725.GA12641@uio.no> To: Steinar H. Gunderson X-Mailer: Apple Mail (2.1084) Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] Best practices for paced TCP on Linux? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Apr 2012 11:54:38 -0000 Hi Yep - you might well be right. I first fell across this sort of thing = helping the guys with the ATLAS experiment on the LHC several years ago. The issue, as best as we could capture it - we hit "commercial = confidence" walls inside network and manufacturer suppliers, was the the following. The issue was that with each "window round trip cycle" the volume of = data was doubling - they had opened the window size up to the level where, = between=20 the two critical cycles, the increase in the number of packets in flight = were several=20 hundred - this caused massive burst loss at an intermediate point on the = network. The answer was rather simple - calculate the amount of buffering needed = to achieve say 99% of the "theoretical" throughput (this took some measurement as = to exactly what=20 that was) and limit the sender to that. This eliminated the massive burst (the window had closed) and the system = would approach the true maximum throughput and then stay there. This, given the nature of use of these transfer, was a practical = suggestion - they were going to use these systems for years analysing the LHC collisions at = remote sites. Sometimes the right thing to do is to *not* push the system into its = unpredictable region of operation. Neil On 6 Apr 2012, at 22:37, Steinar H. Gunderson wrote: > Hi, >=20 > This is only related to bloat, so bear with me if it's not 100% = on-topic; > I guess the list is about the best place on the Internet to get a = reasonble > answer for this anyway :-) >=20 > Long story short, I have a Linux box (running 3.2.0 or so) with a = 10Gbit/sec > interface, streaming a large amount of video streams to external = users, > at 1Mbit/sec, 3Mbit/sec or 5Mbit/sec (different values). = Unfortunately, even > though there is no congestion in _our_ network (we have 190 Gbit/sec = free!), > some users are complaining that they can't keep the stream up. >=20 > My guess is that this is because at 10Gbit/sec, we are crazy bursty, = and > somewhere along the line, there will be devices doing down conversion = without > enough buffers (for instance, I've seen drop behavior on Cisco 2960-S = in a > very real ISP network on 10->1 Gbit/sec down conversion, and I doubt = it's the > worst offender here). >=20 > Is there anything I can do about this on my end? I looked around for = paced > TCP implementations, but couldn't find anything current. Can I somehow = shape > each TCP stream to 10Mbit/sec or so each with a combination of SFQ and = TBF? > (SFQRED?) >=20 > I'm not very well versed in tc, so anything practical would be very = much > appreciated. Bonus points if we won't have to patch the kernel. >=20 > /* Steinar */ > --=20 > Homepage: http://www.sesse.net/ > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat