From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 557A5200034 for ; Wed, 22 Jun 2011 12:44:47 -0700 (PDT) Received: by iyi12 with SMTP id 12so1503373iyi.16 for ; Wed, 22 Jun 2011 13:11:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=5dMT9/C4kmq3MiQwatPkInm3fJ9fx/swQZAsHBDtY+Q=; b=VWkKxQOpB4JkK24Lw2saE6LH4UhTyh7JmqV6CvMw4mVn5dGQp2CCtpr+njvYOSTjnM bqHCiqrKYmjYkpF++HvIHz21juwpSKOWuzQugykDZtUX3l76IAiG62RNei3SzO3hBQpD TRZ4mjGdXHnzmxhZeg64DSTUkNf+UY9XF8P5w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=XG0/kP3Iowl61cI16M6WLKXiZZcGzH9GzqDORjXTRsH02CjOkvhgLDoHFprw5v5i8i Qc5jlpC0uWBePH3MuCksLrj8+mnAfESCAvhfJFHFOJ/mr8sXyJvrip9HDTbD8KfuxxaM Z3h8keqI1pFLbilMDA1Zv2J1ZLB8fqSx1rMHs= MIME-Version: 1.0 Received: by 10.42.29.131 with SMTP id r3mr1133405icc.377.1308773516999; Wed, 22 Jun 2011 13:11:56 -0700 (PDT) Received: by 10.231.13.76 with HTTP; Wed, 22 Jun 2011 13:11:56 -0700 (PDT) In-Reply-To: References: Date: Wed, 22 Jun 2011 14:11:56 -0600 Message-ID: Subject: Re: Mice [was Re: QoS for system critical packets on wireless] From: Dave Taht To: Justin McCann Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat-devel X-BeenThere: bloat-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Developers working on AQM, device drivers, and networking stacks" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2011 19:44:47 -0000 On Wed, Jun 22, 2011 at 1:39 PM, Justin McCann wrote: > On Wed, Jun 22, 2011 at 11:17 AM, Dave Taht wrote: >> The biggest fallout of the diffserv work I was trying was observing >> that most packets fell into one of 3 buckets: >> >> 1) System control and 'some tiny animal' are < less than 1% of all packe= ts. 'some tiny animal' >> includes a bunch of messages like ARP, NTP, UDP, and most of the icmp6 >> portion of the stack, in what I'm doing currently. 'some tiny animal' ar= e >> desperately needed for the network to continue to function. >> >> 2) unresponsive streams and udp (varying, but generally pretty low) >> 3) responsive tcp streams (closer to 99%) > > Dave, > > I want to avoid hijacking your thread, because I think the QoS issues > you're raising are important, but I think you've misused the term > 'mice'. Mice are short-lived TCP flows (see > http://www.google.com/search?q=3Dmice+elephants+tcp), not these > control/management-related low-bandwidth protocols. Obviously the > control/management protocols need priority on the network. "Mice" > really belongs in category 3), which often gets broken down into 3a) > mice and 3b) elephants. EXTREMELY VALID POINT. I utterly agree that I have been mis-using the term 'mice' in this context. I will stop immediately to end the confusion. won't even qualify by saying TCP Mice, gotcha. I have to stop talking to myself more often.... I kind of started extending the term because I'd also lumpled syns, synacks, and fins into the tiny packets that are "often useless and even detrimental to shoot" catagory. Synack optimization is actually a "feature" of many cable modems... and fins have been a problem going back YEARS where many providers simply drop the connection instead of waiting for a correct close. I further make a distinction between control packets and things like DNS/NTP and VOIP as per the above.... While I'm liking using as much Diffserv terminology as possible (CS6 works fine for 'control' packets (some of icmp6 fits into that catagory too), EF works for voip, I really need a word for DNS/NTP and maybe ARP and DHCP that fits the facts somewhat better. They really aren't BE services anymore... sooo.... ***** We need another animal or two, smaller than a mouse, yet required for life, to use as a analogy... e coli? (nah....) Need something with a good back-acronym... lots of animals to choose from.... ???? Ideas? **** The problem with lumping arp into CS6 is that it can and should be rate limited elsewhere. Same for DHCP. Similarly CS6 includes BGP which can lead to enormous transfers.... NTP could go into CS6, except that it's more EF other sorts of UDP are generally small flows, with vpns being that particular udp elephant. So I just want a single word for these sorts of packets that can compete with the TCP mice and elephant discussion... because in forgetting about them, we've entered a world of hurt. > > To resurrect an ancient thread, what Kathie Nichols was referring to > is a more sinister problem: short-lived TCP flows make up a lot of the > *number of flows* in the Internet--- think of all the little JS, CSS > and image pulls a web page causes--- but not the majority of bytes. > You can make long-lived high-bandwidth flows behave without starvation > using ECN and the like, and a lot (most?) of AQM research has focused > mainly on that. Yes I'd understood that, and unfortunately extended it past it's original meaning in several weeks of talking to myself... > The problem is, how do you handle tons of flows which each transmit > just a few packets? Head-dropping one or more packets in a > high-bandwidth flow using SACK won't cause that flow much of a > problem. However, if your queue is full of packets from many different > flows that each consist of just a few packets, dropping *even one* > might cause that flow to hit a retransmission timeout. The overall > connection-completion latency (SYN to FIN) goes up significantly, > which is especially bad if the connections are done serially. > > When you have a relatively small number of high-bandwidth flows, > dropping packets can quickly change the fill rate at the bottleneck > queue. When you have a lot of small flows, dropping packets from only > a few flows doesn't make much difference--- each flow doesn't > contribute as much to the overall sustained rate, so it can't reduce > it by much either. > > I'm not really read up on all the various queueing disciplines. Maybe > when you have a large number of flows in your queue, you need to drop > more packets (from many of them) to get a corresponding drop in the > fill rate. Paying attention to IP-to-IP flows vs. TCP/UDP flows may > also make up the difference when you've got a lot of separate TCP > connections. > > I think most consumer-ISPs' approach to elephant flows is to simply > rate-limit them after a period of time. So much for that rate you > thought you were paying for. > > =A0 =A0 =A0Justin > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://the-edge.blogspot.com