From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-23-ewr.dyndns.com (mxout-048-ewr.mailhop.org [216.146.33.48]) by lists.bufferbloat.net (Postfix) with ESMTP id C3D662E011F for ; Thu, 10 Mar 2011 17:15:44 -0800 (PST) Received: from scan-22-ewr.mailhop.org (scan-22-ewr.local [10.0.141.244]) by mail-23-ewr.dyndns.com (Postfix) with ESMTP id F1F54411CC for ; Fri, 11 Mar 2011 01:15:42 +0000 (UTC) X-Spam-Score: -1.0 (-) X-Mail-Handler: MailHop by DynDNS X-Originating-IP: 209.85.215.43 Received: from mail-ew0-f43.google.com (mail-ew0-f43.google.com [209.85.215.43]) by mail-23-ewr.dyndns.com (Postfix) with ESMTP id 0770B40B17 for ; Fri, 11 Mar 2011 01:15:37 +0000 (UTC) Received: by ewy20 with SMTP id 20so934597ewy.16 for ; Thu, 10 Mar 2011 17:15:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:content-type:content-transfer-encoding :subject:date:message-id:to:mime-version:x-mailer; bh=wfZ6n4EU+L4QuD+XpJmAE9FhnGut658oNsDWHtvWtqY=; b=qEZPHr6Eu8y1FwX5ucX1G0wstntw9/BijRMP3vFH8Rnik0afEqgB0me8B/Cf/G7lmr x5Ke/gKzH1w96kBRhElhyKl8gc/eRCosHii6q/kFwOGhRSQhnKtLnV3NKACWlAKJ/yBr oSuidFX+O2F03b0/eq82N+/yJtMPHwNNH9Ts8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:content-type:content-transfer-encoding:subject:date:message-id :to:mime-version:x-mailer; b=jfbxl0V4xkTdBceDOma7hxE100hGELwV9mnM8KT2npVkUJH1BjDEtBUq5nSo1NMECx vfrfltWWW4U8HwU8xtxD96t1X74loEj+ItBMQbg8gkd+thldoiq/v5hqsHP5P66H6BJA apVwpTdEqKIAHrsPh2YvqOF472vjrhGhy5PfI= Received: by 10.213.13.74 with SMTP id b10mr5325742eba.72.1299806137178; Thu, 10 Mar 2011 17:15:37 -0800 (PST) Received: from [192.168.239.42] (xdsl-83-150-84-172.nebulazone.fi [83.150.84.172]) by mx.google.com with ESMTPS id t50sm3002494eeh.0.2011.03.10.17.15.36 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 10 Mar 2011 17:15:36 -0800 (PST) From: Jonathan Morton Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Fri, 11 Mar 2011 03:15:35 +0200 Message-Id: <1464EA88-F053-4EFE-8148-8B44585B9999@gmail.com> To: bloat@lists.bufferbloat.net Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) Subject: [Bloat] A hard ceiling for the size of a drop-tail queue X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Mar 2011 01:15:45 -0000 There are two mechanisms, in TCP and IP respectively, which suggest = theoretical limits to the acceptable size of FIFO queues (in routers, = modems and other places): 1) The "Initial RTO" value in TCP is 3 seconds, or exceptionally 2.5 = seconds. This suggests a hard maximum of 2 seconds' buffering in order = to avoid needless retransmits during TCP setup. 2) The initial TTL of IP packets is typically 64 hops, and an assumption = in TCP is that a packet will not live in the Internet for longer than = 120 seconds. This also strongly suggests a buffering limit of about 2 = seconds per hop. Admittedly, such buffering is only likely to occur in = the bottleneck link, but 30 seconds RTT has been observed in the wild = and this is already uncomfortably close to 120. The next question is, of course, how many bytes is 2 seconds of = buffering? This depends on the connection type. For a full-duplex = Ethernet or optical link, 2 seconds is a lot of data and so such a limit = is almost meaningless. For an analogue modem or similar "thin" and = stable link, the limit is usually easy to calculate, yet many = professional-grade devices have buffers considerably larger than this. For a shared-media connection, such as bus-type Ethernet or 802.11, such = a naive calculation is very dangerous, because the throughput of the = connection cannot be reliably predicted, especially under congested = conditions. In fact, wireless protocols have several positive-feedback = mechanisms which virtually guarantee that once severe congestion occurs = (as in a typical large conference room), the network will collapse (ie. = goodput will approach zero) and will remain in the collapsed state until = the load is removed completely. Most of these mechanisms are explicitly = not present in classic Ethernet, which will continue to function = usefully (if minimally) in a congested state, and recovers as soon as = the load is reduced. NB: I am writing this from a relatively high-level understanding of how = 802.11 works. I may therefore miss some details. 1) A radio transceiver cannot detect a collision once it has begun to = transmit - it goes deaf. So if two transmissions overlap, it is likely = that both colliding packets will be lost. Furthermore, a transceiver = cannot reliably detect another transmission even before starting to = transmit, because it may be audible to the base station but not itself. = This latter limitation is not present in Ethernet. 2) There is a distinctly finite number of channels available to transmit = on (especially in the usual 2.4 GHz band), and in 802.11 (especially = newer versions) these channels overlap to some degree. The more radios = are transmitting simultaneously on one band, the greater the noise level = seen by other radios sharing the same band, even if they are operating = outside each others' normal ranges. Obviously, overlapping = transmissions due to collisions will increase the background noise = level, which will in turn lead to a greater incidence of collisions. 3) Increased noise also causes transceivers to step down in speed. This = reduces the capacity of the link, increases the time needed to transmit = a packet, and increases the noise energy generated per collision. These = also increase the probability of a given packet colliding. This in turn = causes further step-down in speed. 4) Once the goodput has deteriorated to the extent that some RTTs are = over 3 seconds - at which point the experience is already qualitatively = slow - TCP will retransmit the setup packets, further increasing network = load. DNS has a similar retry mechanism, so applications which have not = yet reached the TCP setup phase are similarly affected. At this point = it is very likely that every device in the airspace is constantly = attempting to transmit something, yet is deafened to the correct = opportunities to transmit without colliding because of the noise from = everyone else also getting it wrong. In the early stages of the above sequence, advanced MIMO base stations = can mitigate the problem slightly by being able to receive packets from = more than one transmitter at once, provided they are in substantially = different directions (which is the most likely deafening scenario). In = the later stages this does not help because most collisions will involve = multiple colliders, some of which will be in similar directions and thus = indistinguishable to the MIMO transceiver. So how many bytes is a 2-second queue on a wireless network? It can be = no more than the *minimum* link speed (in bytes/sec) divided by the = *maximum* number of nodes. This allows a 100% overhead for framing and = congestion avoidance. For 802.11 the minimum link speed is 1Mbps =3D = 100KB/s, so with 100 nodes (not a very large conference) the buffer = should be no more than 1KB - which is less than one full packet. And = since there are only about 3 non-overlapping channels in the 2.4GHz = band, then even if you spread those 100 nodes over 3 base stations, you = only get 3KB =3D 2 packets of buffer space. (And this includes the base = station!) So I'm surprised that these conference networks ever function at all. - Jonathan