From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-gw0-f43.google.com (mail-gw0-f43.google.com [74.125.83.43]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id C2D6D200034 for ; Wed, 22 Jun 2011 12:12:43 -0700 (PDT) Received: by gwj21 with SMTP id 21so651160gwj.16 for ; Wed, 22 Jun 2011 12:39:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to:cc :content-type; bh=Ax9h8Gs2hWP0zXTr3A0FTrBxWdvrcgd/tb1Vazql7F4=; b=gypPIh8tv7katIn8m3QUrxDuXAyrLZDEwvmN3VS6kx6UKTktS1dkBp41ShELXM9KQn xBlpQsIizRwNScMevjFElJZjfVqVGx2cXw3VwjvRzZ8+jXYKxhk3MoDyZ4hBJcibYD6/ dXYsdwd+7lWOaZ6ZVdqy44ArUjRdCgMYyTARc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=URqms02F/NdMgSB95glz1rBMdYcBlMiaxQPZEv7qpF9T1cH2ceIOHVD+x8FW9BOrXv k/piOrS4L4hBrToT4Y4AGpCTVegsMpkGunAK02drHaI0GUn7Iq8HQ4lR7RdF9agUApLX 0iTDDghEHGstssGUI7sk575XcEPbjEc4PKqNw= MIME-Version: 1.0 Received: by 10.236.136.164 with SMTP id w24mr1880851yhi.171.1308771592335; Wed, 22 Jun 2011 12:39:52 -0700 (PDT) Received: by 10.147.34.13 with HTTP; Wed, 22 Jun 2011 12:39:48 -0700 (PDT) Date: Wed, 22 Jun 2011 15:39:48 -0400 Message-ID: Subject: Mice [was Re: QoS for system critical packets on wireless] From: Justin McCann To: Dave Taht Content-Type: text/plain; charset=ISO-8859-1 Cc: bloat-devel X-BeenThere: bloat-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Developers working on AQM, device drivers, and networking stacks" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2011 19:12:44 -0000 On Wed, Jun 22, 2011 at 11:17 AM, Dave Taht wrote: > The biggest fallout of the diffserv work I was trying was observing > that most packets fell into one of 3 buckets: > > 1) System control and 'MICE' are < less than 1% of all packets. Mice > includes a bunch of messages like ARP, NTP, UDP, and most of the icmp6 > portion of the stack, in what I'm doing currently. Mice are > desperately needed for the network to continue to function. > > 2) unresponsive streams and udp (varying, but generally pretty low) > 3) responsive tcp streams (closer to 99%) Dave, I want to avoid hijacking your thread, because I think the QoS issues you're raising are important, but I think you've misused the term 'mice'. Mice are short-lived TCP flows (see http://www.google.com/search?q=mice+elephants+tcp), not these control/management-related low-bandwidth protocols. Obviously the control/management protocols need priority on the network. "Mice" really belongs in category 3), which often gets broken down into 3a) mice and 3b) elephants. To resurrect an ancient thread, what Kathie Nichols was referring to is a more sinister problem: short-lived TCP flows make up a lot of the *number of flows* in the Internet--- think of all the little JS, CSS and image pulls a web page causes--- but not the majority of bytes. You can make long-lived high-bandwidth flows behave without starvation using ECN and the like, and a lot (most?) of AQM research has focused mainly on that. The problem is, how do you handle tons of flows which each transmit just a few packets? Head-dropping one or more packets in a high-bandwidth flow using SACK won't cause that flow much of a problem. However, if your queue is full of packets from many different flows that each consist of just a few packets, dropping *even one* might cause that flow to hit a retransmission timeout. The overall connection-completion latency (SYN to FIN) goes up significantly, which is especially bad if the connections are done serially. When you have a relatively small number of high-bandwidth flows, dropping packets can quickly change the fill rate at the bottleneck queue. When you have a lot of small flows, dropping packets from only a few flows doesn't make much difference--- each flow doesn't contribute as much to the overall sustained rate, so it can't reduce it by much either. I'm not really read up on all the various queueing disciplines. Maybe when you have a large number of flows in your queue, you need to drop more packets (from many of them) to get a corresponding drop in the fill rate. Paying attention to IP-to-IP flows vs. TCP/UDP flows may also make up the difference when you've got a lot of separate TCP connections. I think most consumer-ISPs' approach to elephant flows is to simply rate-limit them after a period of time. So much for that rate you thought you were paying for. Justin