From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-fx0-f43.google.com (mail-fx0-f43.google.com [209.85.161.43]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id BBE4B201A5A for ; Mon, 30 May 2011 08:41:01 -0700 (PDT) Received: by fxm3 with SMTP id 3so4206969fxm.16 for ; Mon, 30 May 2011 08:57:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:references:in-reply-to:mime-version :content-transfer-encoding:content-type:message-id:cc:x-mailer:from :subject:date:to; bh=oowd6itxsGBqv1c/fAj9hvfyFaFm2YX6Zh2fGLmh4a8=; b=UWk3J5qAcpCly+DNfkCTA2AfP1/Ak7I1WIC04tf7rTJbyldLjP8B5SPgbOI9hCdxxQ jdTgdIbAc6TaUJorxjEQp9HFgQPJ+hC+WuqB/zIm1jMMwxWfXWmsQK2o9uW7abyOo8jB ianbA9yG31Y0kMoWI9nCAYoHVzeqWEbmD1C68= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=references:in-reply-to:mime-version:content-transfer-encoding :content-type:message-id:cc:x-mailer:from:subject:date:to; b=VeJYM+Gj3X/iqiuvn4jbU0WFgVhyNqIusCuW2qP0L+a4Vq7icNYugMTy0vkWWRF1Ew eDu7zCnw12y8tvdGop2A+yioNw8VIe1YpIZ+l1NPgzh9k10/9vkCaDBC935EM+ExhLNR Q7Paov63yWanP2m5zYdRbUm37KYbroJBddTT4= Received: by 10.223.96.6 with SMTP id f6mr1575399fan.119.1306771035687; Mon, 30 May 2011 08:57:15 -0700 (PDT) Received: from [87.93.186.37] (87-93-186-37.bb.dnainternet.fi [87.93.186.37]) by mx.google.com with ESMTPS id b25sm786171fab.28.2011.05.30.08.57.09 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 30 May 2011 08:57:14 -0700 (PDT) References: In-Reply-To: Mime-Version: 1.0 (iPhone Mail 8C148) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Message-Id: <0E55287D-41D8-4CE7-8AD8-493A874B80D7@gmail.com> X-Mailer: iPhone Mail (8C148) From: Jonathan Morton Date: Mon, 30 May 2011 18:57:02 +0300 To: "George B." Cc: "bloat@lists.bufferbloat.net" Subject: Re: [Bloat] philosophical question X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 May 2011 15:41:02 -0000 If most of your clients are mobile, you should use a tcp congestion control a= lgorithm such as Westwood+ which is designed for the task. This is designed t= o distinguish between congestion and random packet losses. It is much less a= ggressive at filling buffers than the default CUBIC.=20 Your main bottleneck even at 2Gbps is at the uplink to the ISP. That is wher= e you need an AQM capable router. You have no control over what happens furt= her into the Internet except by turning on ECN. IMHO that is reasonably safe= already and more people should do it, but you would be quite justified in r= unning trials and listening for trouble.=20 What ECN probably needs is a statement from several major players - that is R= ed Hat, Canonical, Linus, Apple, Microsoft - that they will unilaterally tur= n on ECN by default in releases and updates after some flag day. It has, aft= er all, been in RFC and implemented for ages, so any remaining broken networ= ks that actually block ECN packets really have no excuse. Stripping ECN is a= slightly less serious problem which will be easier to address afterwards.=20= If your internal bottleneck is a single dumb switch which supports PAUSE, yo= u shouldn't have much trouble and a basic AQM such as SFQ on your servers ma= y be sufficient.=20 The key to knowledge is not to rely on others to teach you it.=20 On 30 May 2011, at 18:29, "George B." wrote: > On Mon, May 30, 2011 at 5:25 AM, Dave Taht wrote: >>=20 >>=20 >> On Sun, May 29, 2011 at 10:24 PM, George B. wrote: >>>=20 >>> Ok, say I have a network with no over subscription in my net. >>=20 >> I'd love to see one of those. Can I get on it? >=20 > Well, we currently have the potential for some microburst oversub > inside the data center but not too much of it. I can take a 48-port > GigE switch and have 40G of uplink but the switches aren't fully > populated yet. Bottlenecks are currently where we might have 25 front > end servers talking on GigE to a backend server with 20G. So some > potential for internal microburst oversub but that's beyond the scope > of this discussion. >=20 >>>=20 >>> I have >>> 10G to the internet but am only using about 2G of that. This is the >>> server side of a network talking to millions of clients. The clients >>> in this case are on "lossy" wireless networks where packet loss is not >>> an indication of congestion so much as it is an indication that the >>> client moved 15 feet behind a pole and had poor network connectivity >>> for a few minutes. >>>=20 >> Or is using multicast. >=20 > Multicast is a fact of life with which one is going to have to learn > to live. Better to somehow get the gear handling it in a better > fashion, in my opinion. >=20 >>> The idea being that in today's internet, packet loss is not a good >>> indication of congestion. Often it just means that the radio signal >>> has been briefly interrupted. What I need is something that can tell >>> the difference between real congestion and radio loss. ECN seems to >>> be the way forward in that respect. >>>=20 >> Yes. When it works. Which is rarely. >=20 > I have enabled ECN (been following various bufferbloat discussions for > a while) on a couple of machines and also my own machine (my own in > order to see where it might cause any problems browsing) without any > problems so far. "Back in the day" when ECN first came out on Linux, > it was enabled by default and caused all sorts of issues with sites > that simply drop packets with either/any of the ECN bits set. So far > there haven't been any issues that I have run into with ECN set on my > Windows laptop. Once I am convinced that setting that those bits > isn't going to cause problems, I will roll that out in a more general > fashion. But if networks upstream from us clear those bits anyway, I'm > not convinced what difference it will make. >=20 > There is also one fairly small subnet in the overall network where I > have enabled "random-detect ecn" with a policy map on a potentially > oversubscribed link. But that is the only router in the network that > even supports ECN. I have sent an inquiry to the manufacturer of the > rest of the gear about supporting ECN with their WRED implementation > but haven't heard anything from them on the subject. >=20 >>> But assuming my network, as a server of content is not over >>> subscribed, what would you suggest as the best qdisc for such a >>> traffic profile? In other words, I am looking at this from the server >>> aspect rather than from the client aspect. >>>=20 >>=20 >> Ah, ok. This was discussed in this loooong thread: >>=20 >> https://lists.bufferbloat.net/pipermail/bloat/2011-March/000272.html >>=20 >> Some form of fair queuing distributes the load to the ultimate end nodes >> better. >=20 > Ok, as we are using Linux (mostly) for the servers talking to the > clients, it shouldn't be much of an issue to put into place. Thanks > to the pointer to the thread and I will watch as things develop and > see how things go. >=20 >> As for which packet scheduler to choose for that? Don't know, I'm just >> trying to get to where we can actually test stuff on the edge gateways at= >> this point. >=20 > Yeah, what I am most interested in are things like smart > phones/laptops/tablets and not necessarily on WiFi but also on 3/4g > networks. Those things are pulling a lot of traffic these days and the > network can be lossy at times. =46rom my own analysis of traffic > captures, it is fairly easy to see when a device that is "on the move" > changes cell towers. You get a burst of resends and often some out of > order packets and then things settle down for a while. This isn't so > big of a deal if you have only a few mobile clients but sites that > cater to mobile content might have millions of such clients connected > at any given time with many of them in a state where they have > marginal connectivity or are in the process of moving between towers. > So the TCP notion that "packet loss =3D=3D congestion" doesn't apply in > those networks. With those, packet loss is just packet loss and > shouldn't be treated as congestion. This is why I think it is so > important to get ECN working across the Internet. But even with ECN > capable end points, if the routers in the middle are not capable of > using ECN to signal congestion and simply drop packets, there is > always a question of why the packet was lost. >=20 > We need to hammer on our vendors a bit and get them properly > supporting ECN to signal congestion on ECN aware flows. >=20 >> Dave T=C3=A4ht >=20 >=20 > Thanks, all! >=20 >=20 >=20 > g > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat